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MPRS in support of the decisions that they make, and not as the basis for such decisions. 
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EXECUTIVE SUMMARY 

As we enter the 21* Century, the Department of Defense (DoD) finds itself facing 
a significant personnel management crisis. Despite a thirty percent reduction in 
manpower needs over the last ten years, the military is continually failing to meet its 
recruiting and retention requirements. This failure has created a vice-like effect upon the 
DoD’s capability to properly man and train many of its units, thus contributing towards a 
lower overall readiness state for our Armed Forces. 

This reduced readiness was most recently observed when the United States 
participated in NATO’s intervention in Kosovo. During this Operation Other Than War 
(OOTW), US Air Force and Navy combat pilots were not as effective as was expected. 
Furthermore, US Army aviation units failed to meet training standards for their 
utilization. The stress of multiple deployments showed during this period. The best 
representation of this fact was that two Army Divisions were labeled “non-mission 
capable” (NMC). The Congress is now investigating whether or not the military is 
capable of meeting its requirements (to fight and win two simultaneous Major Regional 
Crisis’ (MRCs). 

The readiness problem might not be so stark 1f OOTWs were few and far 
between. Unfortunately, the frequency of such missions has continued to increase since 
the Gulf War (the United States Army, Europe (USAREUR) alone has seen a 500% 
increase in OOTW [Ref 23]). In November 1999, Cable New Network (CNN) reported 
that almost two-thirds of voters support the continued use of the military for missions 
such as Kosovo [Ref 25]. This means that our civilian leadership will most likely 


continue to utilize the Armed Forces for OOTW missions. 
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However, deployments have their personnel cost. They increase the amount of 
time that service members are away from their families. When the OOTW includes the 
possibility of combat, family stressors increase dramatically. The result is an additional 
negative factor influencing retention. When this is combined with other factors such as 
the booming civilian economy, the perceived decrease in retirement benefits (especially 
medical), and quality-of-life issues, the combinatory effect is dramatic. 

The problem is amplified by the impact of the information revolution; the 
widespread use of the Internet by our service members keeps them very well informed 
about their non-military employment opportunities. The result is that the number of 
service members leaving the service continues to rise. 

In October 1999, The Military Times reported that more than half of the enlisted 
force intended on departing the military as soon as possible (the Navy was highest with 
75%). The commissioned officer side of the survey showed that at least 33 percent of all 
officers intended to leave (the Army was highest with 53%) [Ref 13]. Many of those 
who plan to get out are those with critical skills such as computer specialists, technicians, 
and aircraft pilots. As of July 2000, 23 of the Army top 25 most cntical MOS were 
computer operators or technicians. These Military Occupation Specialties (MOS) are 
found in Table 2.5 [Ref 24]. 

A significant portion of the current crisis stems from DoD personnel management 
decisions made during the post-Cold War “drawdown” of the 1990s. The drawdown 
reduced the active force from 2 million down to 1.4 million service members. When the 
plans for the drawdown were being created, the American economy was believed to be in 
a state of recession. Under that belief, it was thought that a mass involuntary separation 
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would simply pour 600,000 of the least qualified service members into Ше 
unemployment lines, thus making the economy even more depressed. Therefore, 
Congress authorized monetary incentives for early retirements and voluntary discharges. 
The incentives included lucrative deals such as the Voluntary Separation Incentive (VSI) 
and the Special Separation Benefit (SSB). 

The target of the incentives was still the lower-qualified service members, but 
these plans would allow them to re-enter the work force gradually (as they had money in 
their pockets), and would allow many of them to start up their own businesses; things that 
should improve, and not hurt the economy. Unfortunately, the economy was no longer in 
a state of recession, and the result was disastrous. Many of our best-qualified personnel 
reviewed their situations, and found it to their personal advantage to depart the service; 
these personnel not only departed directly into high-paying civilian jobs, they were paid 
by the government to do so. 

The perception of military employment also changed during this time; many 
civilians gained the perception that the military was not recruiting anymore, and those 
that left the military for civilian employment were spreading the perception that military 
careers were no longer viable. The military actually had to advertise that we were still 
hiring! 

As the services were only using linear programs to study retention behaviors, and 
as we were trying to eliminate over half of a million positions, the dramatic change in 
retention behaviors was not detected until it was too late. During the last half of the 
1990’s, the military has found itself trying to regain this lost ground. To date, the Army 
is actively trying to convince VSI and SSB soldiers to come back [Ref 26]. 
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Had military manpower experts properly understood the retention factors they 
were facing in the early 1990’s, it 1s easy to see that they would have made many 
decisions differently. But how could they have known, and how do current and future 
decision-makers stop from making the same kinds of mistakes? Furthermore, how can 
they know which factors they should manipulate (of those they can)? 

Although we can blame technology for amplifying this problem, we can also look 
to technology to help solve it. This thesis takes advantage of today’s dramatically 
increased processor speeds, modern object-oriented programming, and non-linear Agent- 
based techniques in order to provide a non-linear supplement to the current linear model 
only approach. 

The model is called the Military Personnel Retention Simulator (MPRS). The 
MRS is designed to help personnel managers fully understand all possible ramifications 
of their decisions. While it cannot predict the future, it can show the scope of possible 
futures through the use of exploratory modeling. When used as a supplemental analysis 
tool, this model will be able to warn personnel strength managers of problems they may 
encounter. 

The model utilizes computerized objects (“Agents”) to create a Multi-Agent 
System (MAS). The Agents emulate the behaviors of a specific critical military 
community. The MAS changes the environment that the Agents “live” in. Thus, 
personnel managers can observe what large-scale “group” retention behaviors may result 
after they implement a particular policy. 

The model comes with a Graphical User Interface (GUI) to help guide the user 
through the process. The GUI provides a demonstration of how the MPRS works. It also 
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solicits input into how the user would like to run his/her own simulations. This 
eliminates the necessity of each user having to be an expert in the Java Programming 
Language in order to alter what specialty 1s studied, and the strengths of each factor used 
in the study. 

The model is exploratory in nature, and therefore not predictive. The user is 
urged to use this model in support of the decisions he/she makes; it should not be used as 


the basis for such decisions. 
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I. INTRODUCTION 


“Whereas the purpose of induction is to find patterns in data and 
that of deduction is to find consequences of assumptions, the purpose of 
agent-based modeling is to aid intuition.” --- Robert Axelrod 


А. THESIS STATEMENT 


A multi-agent system simulation in support of personnel retention analysis 
provides personnel managers with a thinking tool to help them better understand and 
forecast the retention behaviors of service members. 


B. THESIS MOTIVATION 


The Naval Postgraduate School has long had a reputation for quality research in 
both the areas of military retention, and of artificial intelligence. Over the last ten years, 
significant changes have occurred in both fields. One of the driving forces behind the 
changes in both areas is the Technological Revolution. Like the Industrial Revolution, 
the most significant changes involve how quickly things get done. 

1. Retention 

In the area of military retention, service members can now gain access to 
unlimited amounts of information about the civilian job market, and they are finding that 
there are numerous opportunities for them, regardless of their military occupation 
specialty (MOS). Those service members with technical backgrounds have even greater 
opportunities. The result has been that service members are departing the service in 
much larger numbers, both before and after they are retirement eligible. 

Many service members are opting to leave the military before retirement because 


they see their financial opportunities as being greater getting out immediately, rather than 
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after earning a retirement check. Other service members are retiring, but they are not 
staying in the service until they reach their retention control point (RCPs), which exceed 
the 20-year benchmark by up to ten years. These changes have caused significant 
challenges to the linear retention models that the services use; even the Air Force (long 
known for its high retention rates) has been caught off guard by technical and mechanic 
MOS personnel departing in extreme numbers. These drastic changes have presented a 
new challenge to NPS operations analysts. 

2 Artificial Intelligence 

In the area of Artificial Intelligence (AI), processor speeds have increased so 
dramatically, that many applications previously deemed conceivable, but not practical, 
have come to fruition. One such concept is that of using a computerized object to 
represent each part of a large system. The objects (called “agents”) are now being used 
to develop new computer software programs around the world. Agents are appeanng 
everywhere from inside electronic equipment and videogames, to modeling the behaviors 
of insect colonies, to surfing the Internet on behalf of a user so that the user does not have 
to waste personal time doing routine tasks. 

Agent technologies are skyrocketing, and adding new breath into the Al 
community. In 1998, the NPS hired an agent software developer (Professor John Hiles) 
to support agent-based projects and to teach agent-programming techniques. These 
courses have spurred numerous theses to develop, analyzing everything from 
communications to armed helicopter reconnaissance. In addition, major agent-based 
simulation projects are now under way, making the NPS a significant player in agent- 


based research and development. 


3; Using Agents for Operations Analysis 

Midway through the 1990’s, an NPS computer scientist named Professor Michael 
Zyda saw the need for merging Operations Analysis with Computer Science. He 
therefore created a new Academic Group at the NPS entitled “Modeling, Virtual 
Environments, and Simulation” (MOVES). 

The MOVES Academic Group utilizes the academic assets of both the Operations 
Research and Computer Science Departments. This interdisciplinary approach benefits 
both students and professors. The professors benefit by working with each other, and 
learning how to incorporate the “sister” field into their own. One such professor was 
Doctor William Krebs who conceptualized the merging of agents into his field of military 
retention analysis. Professors Krebs and Hiles therefore agreed to co-advise the 
development of this military retention simulator. Six months prior to the release of this 
thesis, Professor Krebs transferred to the Federal Aviation Administration, and was 
replaced by Professor Zyda as thesis advisor. 


С: THESIS SCOPE 


This model is being provided as a thinking tool to use as a supplement to current 
linear models. It is exploratory in nature, and therefore not predictive. The user is urged 
to use this model in support of the decisions that he/she makes, and not as the basis for 
such decisions. If the model predicts behaviors contrary to the user’s expectations, the 
user should take this as a warning sign that environmental conditions may be primed for 


extreme, if not chaotic behavior changes. 


This model is designed to emulate the retention behavior of unique communities 


within the DoD. As such, the user must be intimate with the knowledge about the 
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community that he/she 1s attempting to model; the level of the quality of the information 
input into the model will drive the quality of the output derived from the model. This 
model also calls for the user to interpret the perceptions that the community has about 
pay gaps, and the extent to which the chains of command and the DoD care about the 
individuals in the community; 1f survey information for a specific community can not be 
obtained (from the Defense Management Data Center (DMDC) or other resources), the 
user should make estimates based upon the feedback that he/she receives when dealing 


with members of that given community. 


Although the model is designed to emulate homogeneous sub-communities within 
the DoD, it could conceivably be expanded for larger, more general groups of service 
members. However, the user should note that as the behavioral input goes from specific 


to general, the associated output will become less accurate, and of less use. 


The model is designed to produce outputs that are displayed for the user on the 
DOS Command Line. Analysis of this output is beyond the scope of this thesis and is 
designated as future work. 


D. THESIS OBJECTIVE 


The purpose of this study is to provide military personnel managers an advanced 
computer simulation model that will assist in the study of non-linear retention behaviors 
of critical military professions. My intent is that the MPRS model be used to supplement 
the current top-down structured models in all branches of the Armed Forces for the 


analysis of any critical skill. 


The primary target for analysis is critical commissioned officer specialties from 


within any branch of the Armed Forces. The secondary targets for analysis includes 


critical enlisted and warrant officer specialties. Tertiary targets include larger, non- 


homogenous groups of servicemen from the same career path (commissioned 


officer/warrant officer/enlisted). 


Properly utilized, the MPRS will provide the full spectrum of possible changes in 


trends, thus making the user cognizant of potential problems. This model is designed to 


provide personnel retention decision-makers with a thinking tool that will supplement 


traditional models; it is not meant to be their replacement. 


E. 


THESIS ORGANIZATION 
This thesis is organized into the following chapters: 


Chapter I: Introduction. Identifies the purpose and motivation for creating this 
model. Establishes the objectives for this thesis. Explains the organization of this 


thesis. 


Chapter II: Background. Discusses the background of the current military 
manpower crisis. Discusses the advances in computing power that has allowed 


agent-based simulations to advance. 


Chapter III: Previous Research and Literature. Discusses the research conducted 
in the fields of Military Retention and Complex Adaptive Systems. Discusses 


recently released and relevant papers. Defines key concepts and terms. 


e Chapter IV: Model Development: Describes the development and purpose of the 
model. Identifies file hierarchies and defines each of the twelve major 
programming files. Explains how to obtain a copy of the model, and how to 


properly operate it. Discusses initial data analysis. 


e Chapter V: Conclusions and Future Work. Discusses information and concepts 
learned during the development of the model. Discusses future work to be 


completed by the author, and recommendations for other researchers. 


Il. BACKGROUND 


“If we, as senior leaders, do not take action now to tum this 
around, we may not be able to meet our future requirements.” 
--- General John M. Keane, US Army Vice Chief of Staff, 2000 


A. INTRODUCTION 


General Keane’s above-stated concern was about the 10.6 percent voluntary loss 
rate for captaıns (officer pay grade 3) that existed in the Spring of 2000. This rate has 
continued to grow, and 1s now nearly twice that of the pre-Operation Desert Storm Army 
[Ref 55]. This situation is just a symptom of a larger, DoD-wide problem. 


B. THE RECRUITING AND RETENTION PROBLEM 


Every year, the Defense Manpower Data Center (DMDC) produces a report called 
the “Youth Attitude Tracking Study” (YATS) for the Department of Defense. DMDC 
gathers the information for the study through interviews with young Americans, aged 16 
to 24. The 1999 report shows the continuation of the trend away from military service 
throughout the 1990’s. Although there are a number of reasons for this downward trend, 
the bottom line found in the report 1s that youths are no longer looking at the military as a 


primary source of preparing themselves for the future [Ref 37]. 


The YATS is just one of many DoD sources reporting negative information back 
on the personnel challenges facing the Armed Forces here at the beginning of the 21° 
Century. The military personnel system has thus entered an uncharted era of constant 
change in both the recruiting and retention (R&R) arenas that 1s threatening our military 


readiness posture. 


The constant change in R&R behaviors has caused a personnel crisis unlike any 
that the DoD has ever had to deal with since the implementation of the “all volunteer” 
force. Despite reducing the total number of personnel needed by the active forces by 
over thirty percent, the DoD still finds itself unable to consistently retain the right amount 
and types of personnel that it needs. This problem is further complicated by the DoD’s 
inability to recruit enough new service members to replace its losses [Ref 28].. These 
simultaneous problems act like a vice upon the personnel strength management of each 
service. 

1. Enlisted Recruiting 

The enlisted force is traditionally the focus of recruiting. Unfortunately, YATS 
numbers indicate that the prime target for recruitment (16-to-21-year-olds) has lost 
almost 30 percent of their interest in joining the military since 1989 (going from 17 
percent to 12 percent during the decade). Tables 2.1, 2.2, 2.3, and 2.4 reflect the enlisted 
military recruiting results over the last five years, as reported by CNN іп October 2000 
[8025]. 


U.S. Military Recruitment, 1995-1999 


U.S. Army 


Fiscal year Achieved 


62,967 


73,528 
82,087 
71,749 
68,210 


Source: S. Douglas Smith, Public Affairs Officer, U.S. Army Recruiting 
Command 


Table 2.1: US Army Recruiting, 1995-1999 (from Ref 25) 





U.S. Military Recruitment, 1995-1999 


The Navy's goals are predictions for Congress of how many 
sailors will leave during a given year. Actual recruitment is eventually 
adjusted to match actual attrition. 

Source: Lt. Steve Zip, Deputy Public Affairs Officer, Navy Recruiting 
Command 


Table 2.2: US Navy Recruiting, 1995-1999 (from Ref 25) 





U.S. Military Recruitment, 1995-1999 


Fiscal 





Жар е 5: US Marine Corps Recruiting, 1995-1999 (from Ref 25) 


U.S. Military Recruitment, 1995-1999 


U.S. Air Force 


Fiscal year Achieved 


The Air Staff at the Pentagon annually sets goals for 
recruitment based on attrition, then when those goals are met the 
figures for actual recruitment are adjusted to match them. Therefore, in 
years that are closed on the books, figures for goals and actual 
recruitment are the same. 

Source: Master Sgt. Thomas Clements, Superintendent, Air Force 


| Recruiting Service Public Affairs, U.S. Air Force | 
Table 2.4: US Air Force Recruiting, 1995-1999 (from Ref 25) 





The above tables reflect that the three main services (Army, Navy, and Air Force) 
have all had a significant enlisted recruiting problem during the last two years. The 
Army’s most severe problem was an eight percent failure in 1999; the Navy’s was a 
twelve percent failure in 1998, and the Air Force’s was a five percent failure in 1999. 
The Air Force's failure was significant, in that it 1s normally perceived as the service of 
choice amongst eligible recruits. 

The Marine Corps did not have a recruiting problem during the five-year period 
analyzed; this 1s mostly due to its MOS configuration brought about with its association 
with the Navy. The current enlisted R&R problem appears to be one primarily involving 
special skilled personnel such as computer specialists, mechanics, and technicians. As 
the Marine Corps primarily consists of combat arms MOS, it is able to focus its recruiting 
advertisements on its image as an elite combat force. Table 2.5 (Army Top 25 Critical 
MOS for May 2000) reinforces this concept, reflecting only two combat arms MOS (note 


that Air Defense Artillery crewmembers are considered to be technicians). 


10 


| | Кес. 
Title MUERE. A RA Priority | 
Fire Spt Specialist 
MLRS Crewmwmber 
Patriot FC Op/Maintainer 
Patriot LS Op/Maintainer 
Avenger Crewmember 
Cavalry Scout 
LC Elec Msi Sys Rep 
Network Sw Sys Op 
Mech Xmission Sys Op 
Radio/Comsec Repairer 
Power Gen Eq Repairer 
Chemical Operations Spec 
LT Wheel Vehicle Mec 
SP FA Sys Repairer 
Heavy Wheel Veh Mech 
OH-58D Helicopter Rep 
UH-60 Helicopter Rep 
Petroleum Supply Spec 
Parachute Rigger 
Unit Supply Specialist 
Air Traffic Control 
Intelligence Analyst 
EW/Sigint Analyst 
Non-Commo interceptor 
EW/Sigint Recruit 
Table 2.5: US Army top 25 critical MOS for July 2000 (from Ref 24) 
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2. Enlisted Retention 

Perhaps the only challenge to the military as significant as 115 enlisted recruiting 
problem is its enlisted retention problem. Concern for retention can be seen in the 
communications of the leaders at every level of command, and within each service. 

Concern for enlisted retention appears to be growing strongest within the Air 
Force. In the past, retention was a word that the Air Force normally only used when 
discussing their efforts to keep their pilots. In October 1998, Air Force Magazine 
detailed the problems that the Air Force was beginning to have with its enlisted retention 


[Ref 34]. In this article, Air Force Chief of Staff (General) Michael E. Ryan stated that 
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the problem needed to be addressed seriously, even discussing the concept of reducing 
OPSTEMPO in order to aid enlisted retention. 

Despite the Air Force goal of retaining 75 percent of its first term airmen, it is 
seeing extremely disheartening numbers in their technical-skill MOS, to include air 
traffic controllers at 52 percent and communications-computer system controllers at an 
incredulous 31 percent (see Table 2.6 for more Air Force critical MOS information). It is 
obvious that the close association between these Airmen and their civilian counterparts 
plays a large role in the poor retention rates. The problem appears to be getting worse for 
the Air Force as reflecting in the July 2000 issue of Zhe Times, which reported that the 


Air Force was missing its retention goals in all categories [Ref 35]. 





Top 10 Watch List, Enlisted Retention 
151 


29 
егт erm erm 
Combat Controllers 100% 

1 

1 






Уеаг 





3d 
о 
F-16 crew chiefs 92% 
Airborne BM pers 100% 
8 
0 






Com-computer sys con 42% 
Source: USAF, Fiscal 1998 figures are for the first three 
quarters 










Table 2.6: Air Force Top 10 Critical Enlisted MOS (from Ref 35 ) 


To find the root of the Air Force problem, they polled their service members on 
four subsets of what they called “General Well Being.” Table 2.7 1s the results of that 
poll. Of note, pilots (despite being officers) had the lowest ratings for their perceptions 


of how well they were kept informed by the Air Force (even lower than junior enlisted 


= 
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airmen); they also ranked last in how well they stated their families supported their 
careers (this may be a factor of pilots being the significant combat fighters in the Air 
Force). 

The junior enlisted retention problem is highlighted in these numbers by being 
second behind the pilots in both aforementioned categories, and first in the other two 
categories (“Is the Air Force a good place to work?”, and “Does the Air Force provide a 
good quality of life?") [Ref 49]. Unfortunately, the Air Force was unable to react 
quickly enough to these warning signs, and in 1999 suffered a significant manpower 


setback. 


| Air Force General Well Being 


% Officer % Enlisted % Civilian 


it EA 

USAF isa Seed place to work 81| | | | 
mn ea Te Е 
L "riysspesestame — [n[w wa |- pw [s- 


Table 2.7: Air Force Poll on Attitudes towards service, August 1998 (from Ref 34) 













Similar problems are being found in all other branches of service. The Army is 
attempting to counter this problem by increasing their annual enlisted bonus program 
budget by 64% (from 50 million to 82 million dollars per year). In some cases, the Army 
Is seeing that no amount of additional bonus money will retain soldiers with hot MOS. In 
an attempt to embrace the problem, the Army has started a new program called PaYS 
(Partnership for Youth Success) which provides young Americans hiring preference with 


numerous civilian corporations after serving a tour with the Army [Ref 38]. Although the 


PaYs program does not fix the retention problem, it increases the number of soldiers 
entering the Army, thus providing some relief. 

Another negative retention trend is being seen within the non-commissioned 
officer ranks (senior enlisted). Retiring service members throughout the Armed Forces 
are departing at 20-years at a significantly higher rate. In the past, the military could 
always count on many of these NCO's to stay in until they had.to leave under the RCP 
Program (which allows service members to remain in the military for up to ten additional 
years, based upon their rank). The current trend appears to indicate that retirement- 
eligible NCO's are rushing to civilian employment, much the same as those junior 
enlisted service members who are choosing to ETS. 

3. Officer Retention 

While R&R programs have traditionally been seen as enlisted personnel 
management problems, they are now also becoming problems for officer personnel 
management. Mid-career officers are exiting the military at an alarming rate. In October 
1999, The Military Times reported that over one-third of the officers in every service 
intended on departing the military as soon as possible [Ref 13]. This trend has been 
growing since the end of the cold war. In 1989, only 6.7 percent of Army captains left 
the service voluntanly each year; this rate has increased almost 60% since that time, 
sitting at 10.6 percent for 1999 [Ref 30]. 

While it might be thought that the officer personnel resigning are not the quality 
officers that the military wants to keep, it should be noted that this dissatisfaction has 
now appeared at the US Army’s Command and General Staff Course (CGSC/CSC). 
When Army Chief of Staff General Eric K. Shinseki commissioned a survey of students 
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at CSC, the results were astounding. Although the results of the meetings were supposed 
to be for internal-Army use only, they quickly leaked out, and were spread across the 
Internet via electronic mail. The results showed that even these officers, being primed to 
be tomorrow’s senior-Army leaders, were greatly dissatisfied, and many intended on 
leaving before they become retirement eligible [Ref30]. 

The Navy is also experiencing a significant loss in commissioned officers. A 
1999 Navy survey found that many officers did not want to be put into leadership 
positions because they did not want to be micromanaged. Although the Navy offers a 
50,000-dollar bonus for taking such “Department Head” tours, they were having 
problems getting officers to take them. Another significant reason for avoiding these jobs 
was the workload. While all other branches can count on reserve units to take rotations, 
there are no ships in the Navy Reserves. Navy reservist can only supplement working 
crews on an individual basis, which does not alleviate the additional workload and 
stressors emplaced upon Department Heads [Ref 36]. 

Air Force pilot retention is thoroughly discussed by Captain Marty Gaupp, in his 
thesis completed at the Air Force Institute of Technology (AFIT) [Ref 2]. This specialty 
is going through phases that prove Captain Gaupp’s assumption that a Complex Adaptive 
System (CAS) was appropriate for modeling pilot retention. In October 1998, Air Force 
Magazine also reported extreme changes in retention behaviors that caused the Air Force 
to lower their projected pilot strength by over 30 percent within a few months of their 


original projection! [Ref 34]. 
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4. Officer Recruiting 

The officer-recruiting problem also appears to be getting worse. In July 2000, the 
USA Today reported that for the fifth straight year, both the Army and Navy failed to 
attract enough qualified persons to serve as newly commissioned officers; the report also 
stated that the Air Force was also having sporadic problems with recruiting new officers. 
While the Army has been able to overcome their problem with a short-term solution of 
expanding its Officer Candidate School (OCS, a school which trains enlisted soldiers to 
become Army officers) Program, the Navy’s ОСЗ Program will not be able to make up 
their shortage. 

It should be noted that the Army’s short-term solution to fix their officer 
recruiting problem actually worsens its enlisted retention problem, as it strips the enlisted 
ranks of an additional layer of highly-qualified non-commissioned officers (NCO’s). 
Furthermore, it strips the Army of some of its best future leaders; while these NCO's 
should make outstanding junior officers, most will never become senior officers, as they 
will end up meeting the Army's RCPs, and being forced to retire before they reach senior 
officer ranks. We cannot underestimate the value of having highly qualified NCO's 
advising tomorrows senior-level Army officers. 

The long-term solution for fixing the enlisted recruiting problem is to expend 
more resources on the problem. The military is therefore increasing its spending on 
advertising, the number of recruiting stations, and the incentives that 1t 1s giving out to 
join the military. The military is also dedicating an increased number of service members 


to serve as recruiters. 


The long-term solution for officers provides an extra level of difficulty. During 
the drawdown, the military closed a significant number of Reserve Officer Training 
Corps (ROTC) detachments and sub-detachments. Reopening these detachments is 
significantly more difficult than renting local mall space (where we are trying to emplace 
many of our enlisted recruiting stations). Reopening an ROTC detachment means going 
through the politics of each college campus, many of which do not want an ROTC 
presence. Adjusting the ROTC-recruiting base is not an easy task, and will always lag 
behind the current needs of the military. 

5. The Changing Civilian Environment 

The retention and recruiting situations are actually just symptoms of a larger 
overall problem: The current employment environment in the United States makes 
civilian employment much more attractive than military employment. This environment 
consists of many factors such as pay, duty hours, benefits, and quality of life. 

In the past, the military was very appealing to high school graduates who could 
not obtain a college scholarship. The military was seen as a place where young adults 
could learn skills, earn college money, and pursue adventure. Many college students 
became officers for similar reasons; the military was a good job that provided young 
officers skills and adventure. Many of these service members stayed in the military until 
retirement because the lifestyle was deemed acceptable (and often times more attractive 
than the civilian-sector alternative). 

In the late 1980’s, military employment had an added advantage due to the US 
economy’s entering a recession. After the conclusion of the Gulf War in 1991, all eyes 
were focused on the economy. Both the Executive and Legislative Branches of the 


17 


Federal Government believed that a reduction in force (RIF) was appropriate, and that the 
"peacetime dividend" resulting from the RIF might help the government out of the 
recession. The government therefore created plans for the “drawdown.” 

6. The Drawdown 

Rather than reducing the forces via massive involuntary separation boards, 
Congress deemed it more appropriate to provide incentives to encourage personnel to 
leave the military for civilian employment. Examples of these programs include the 
Voluntary Separation Incentive (VSI), the Special Separation Benefit (SSB), and Early 
Retirement (ER). The hope was that the resulting Armed Forces would consist of highly 
qualified personnel who truly wanted to serve in the military; it was also hoped that those 
lower qualified personnel would take the incentives. What the Congress and military did 
not perceive was that the civilian employment environment had already bottomed out, 
and was in a state of recovery. The civilian sector would end up absorbing all that the 
military wanted to release, and more. 

As the civilian economy “roared back to life," more and more service members 
began perceiving the civilian environment as being more attractive than the military 
environment. Both word-of-mouth, and information provided via the Internet convinced 
many of the military's best service members to depart the service. The economy 
continued to grow throughout the 90's, spurring more and more service members to 
resign or ETS (leave the service at the expiration of their time in service requirement). 
Currently the services are trying to get some of these lost service members back into the 


service [Ref 26]. 


г The Technological Revolution 

One of the most significant reasons for the US economic turnaround has been the 
technological revolution. Events such as the Congress opening up the Internet, and the 
exponential increase in computer technologies have spurred the creation of numerous 
new companies, and the expansion of many older ones. This expansion has driven down 
the civilian unemployment rate to a range that Keynesian Economist previously believed 
to be unobtainable (below 6.5 percent unemployment). Transitory unemployment is now 
almost unheard of, as individuals already have new jobs lined up before they leave their 
old ones (the phenomena is most prevalent in the technologies sector). 

The civilian economy has gotten to the point that there simply are not enough 
employees in many sectors. Technological companies alone need well over 250,000 
more employees, and they see the military as an employment pool to fill many of those 
jobs. Furthermore, these companies directly compete with junior colleges and the 
military for high school graduates. The end result has been that graduating high school 
students now have more options available to them than any group of their predecessors. 

8. The Changing Military Environment 

During the decade of the 1990's, the military environment has also been 
changing; unfortunately, the general perception is that the change has been for the worse. 
Both current and perspective service members are being heavily affected by the 
perception that the military provides an inferior lifestyle than the civilian sector. There 
have been many reasons for this: 

e The number of service members receiving government and civilian relief 
has highlighted the difference in military-to-civilian pay. While FY2000 
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will see a significant pay jump for officers, and FY2001 will see a 
significant pay jump for enlisted, the perception has already been created 
that the military 1s not a good environment for young families, and that the 
military 1s not taking care of its own [Ref 27]. When FY2000 basic pay 
for private (E-1's) 1s divided down into an hourly rate, it equates to $5.37 
[Ref 28], far below what fast food restaurants are offering in most major 
cities. 

The restructuring of active and retired medical benefits in order to save 
money continues to take its toll. Many service members now think that 
their (and their dependents’) medical benefits are lacking. Worse yet, 
these service members are observing their predecessors (military retirees), 
appealing to the government for help in protecting their benefits, and 
launching lawsuits in an attempt to preserve their medical benefits (e.g. 
HR 2966). This situation has even turned up in the political arena, where 
Senator John McCain used it during his unsuccessful bid for the 
presidency in 1999. 

The “Redux” retirement pay plan created a perception that military 
retirement pay was insufficient. Despite the reversal in this policy, the 
negative perception of inadequate retirement pay still exists. It also leaves 
the question in many minds as to whether or not the Redux Program will 
appear again should economic times become harsh again. 

The military caused the reduction of force structure to go slower than the 


reduction in personnel. This caused numerous units to be under-manned, 
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under-trained, and under-equipped. This created a new perception of the 
“hallow military’ that the DoD stated it wanted to avoid. It also created a 
perception that the higher-ranking military leaders were preserving the 
force structure in order to maintain leadership authorizations for 
themselves. 

e The increase in operations tempo (OPSTEMPO) within the military has 
greatly increased since 1991. After conclusion of the Cold War, and the 
US military’s successfully defeating Iraq in the Gulf War, the United 
States has seen fit to use the reputation of its Armed Forces in order to 
exert political influence in numerous “hot spots” around the world. These 
Operations Other Than War (OOTW) have continued to keep the military 
in a high state of deployment. The number of OOTW missions for the 
United States Army, Europe (USAREUR) alone has increased by 500 
percent since the conclusion of the gulf war. The timing of many of these 
deployments has also been a factor, as many have started just before the 
holiday season. 

While it is impossible to exactly predict what the civilian employment 
environment will be like in the future, it is not necessarily the same case for the military 
employment environment. Many of the reasons that the military has become less 
attractive were, and are, controllable. Examples of controllable factors include the 
OPSTEMPO, the perceived civilian-military pay gap, and the perceived concern of local 


chain of commands, and the DoD. 
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The question that should be asked is “To what extent should we focus resources 
on each of these areas in order to exert control over them?” To date, the military is 
focusing its efforts on closing the pay gap with the civilian sector. However, historic 
manpower studies have shown that increased pay is not a good long-term input for 
increasing long-term job satisfaction. The Navy Department Head study supports this 
stand that money alone will not create job satisfaction [Ref 31]. The military must also 
focus on the work environment, and showing that they care for the troops (on both local 
commander and senior leadership levels). 

Had previous military personnel planners been capable of studying all of the 
possible ramifications of their decisions before making them, they might have been able 
to make more accurate decisions. Furthermore, given a heads up, military planners could 
have reacted quicker to policies that were not having the desired affect (such as those 
dictating the drawdown). 

Although the military has had computer models to help predict retention for quite 
some time now, these models are all top-down, linear programs. They study data over a 
long period of time and attempt to assign values to factors based upon this historic data. 
These models do a good job over the long run, but cannot react quickly to sudden, 
extreme changes in the environment. In the case of the drawdown, it took several years 
before we realized that too many people were getting out. The result was that we 
continued to pay people to leave, when at worst, we should have been reclassifying them 


to other MOS within our force structure. 
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The problem with using linear programs to predict human behavior, is that they 
are incapable of adequately representing the extreme group behaviors that can often 
occur. Linear Models assume that outputs will react in accordance with the inputs. 
However, when it comes to human behavior, one additional increment of input could 
have extreme outputs, and extreme amounts of input can have little, none, or a reverse 
effect. 

In the area of military personnel retention, there is currently an additional problem 
when only using linear models. The military environment has changed so drastically 
over the last eight years, that historical data is almost irrelevant. Today’s military 
environment is smaller, and 1s changing constantly as the military and Congress attempt 
to devise the right amount and types of units to have in the Armed Forces. These changes 
cause additional reactions in the behaviors of service members. 

9: Affects of Human Interactions 

In addition to the military system itself, there are other individuals associated with 
the system that can have a significant impact upon the retention behaviors of each service 
member. These individuals include military peers, mentors, civilian friends, and the 
family of each service member. 

Military peers often have significant effects upon each other. These relationships 
can even have a greater impact than that of the chain of command. Often times, informal 
leaders are better able to sway the behaviors of organizations than those formally in 
charge of it. When this is a positive effect, the results can be outstanding, with units 


meeting goals that others only dream about; when it is a negative effect, the results often 
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cause leaders to be relieved (fired), and all involved to develop a bad perception about the 
organization as a whole. 

Although the military does not have a formal mentorship program (beyond the 
rater-ratee system), almost every service member develops a relationship with a senior 
service member that continues past any formal boss-employee relationship. Sometimes 
this is a positive role model that the junior service member wants to emulate; other times, 
the senior service member is either a negative role model, or an example of what not to 
become. When junior service members observe what happens to senior leaders, they 
learn from it, and it has an effect upon their behavior. 

Civilian friends also have an influence upon service members. Most importantly, 
they serve as liaisons between what life is like inside the military versus the civilian 
world. The impact can be considerable if the civilian fnend performs the same type of 
work as the service member (for example, a civilian policeman who is the friend of a 
military/security policeman). These relationships allow direct comparisons between the 
pay, and lifestyles. If the civilian friend 1s an ex-service member, the impact can be even 
greater. 

Perhaps the one group of people that have the greatest impact upon the retention 
behaviors of service members is his/her family. When people initially join the military, 
they normally have very few, if any dependents. As time passes, service members add 
additional dependents, and undergo life changes that have considerable impacts upon 


their retention behaviors: 
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The Sailor who joined the Navy to see the world finds it progressively 
harder to deal with being absent from his young family for six months at a 
time. 

The Marine Officer who married after graduating from Annapolis, finds it 
harder to justify to his equally well-educated wife why they must move 
every one to two years, causing her to reset her career every each time 
such a move occurs. 

The Soldier with an exceptional family member (one with exceptional 
medical conditions) comes to realize that the medical care his child 
receives might be considerably better if he were working for a civilian 
company instead of relying upon CHAMPUS (Civilian Health and 
Medical Program for the Uniformed Services). 

The Air Force mechanic who cannot justify why she must receive civilian 
relief (food stamps, etc.) in order to take care of her four children, when 


her civilian friend 1s making twice as much working for United Airlines. 


The bottom line 1s that as time passes by, the strain of the military lifestyle has a 
considerable impact upon a family. As statistics show that over 60 percent of the force 1s 
married [Ref 36], this impact affects the majority of our servicemen. All too often, this 
impact results in divorce; some reports indicate that the divorce rate for military 
marriages runs aS much as 20 percent higher than the civilian divorce rate [Ref 32]. 
Regardless of whether or not a military marriage ends in divorce or not, the strain of the 
military lifestyle on families continually causes service members to re-evaluate their 


internal cost-benefit equation of continued military service. 
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С. ADVANCES IN COMPUTER TECHNOLOGY 


While changes in the military have been negative, the changes within the 
computer industry have been extremely positive. 

1. Modern Computing 

This model utilizes recent technological and conceptual achievements that were 
unavailable when currently used linear-only models were developed. These 
advancements include the utilization of Java (the programming language of the present 
and the foreseeable future), and the modern concepts of object-onented programming 
(OOP), and Multi-Agent Simulations (MAS). The model takes advantage of today’s 
ever-increasing processor speeds in order to manipulate tens of thousands of objects in a 
parallel environment (a concept unheard of when current models were developed). Users 
should thus note that the model is best run with computers with processing speeds of 700 
Megahertz or more. 

2 The Complex Adaptive System Approach 

The overall concept that drives this model is that of seeing the military personnel 
retention environment as being a Complex Adaptive Systems (CAS). CAS are systems 
that are comprised of many components or subsystems (up to millions) which when 
reviewed as a whole, display a group behavior. These behaviors change over time; 
sometimes they are rigid and unchanging, while others they may seem quite chaotic; 
normally, they fall somewhere in between these two extremes. 

Dr. Brian Arthur of the SFI, and formally of Stanford University supports this 
definition with the paper he presented to the American Economic Association (AEA) 


during their 1994 annual meeting, entitled Bounded Rationality. In this paper he stated 
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that a CAS 1s a system that fluctuates between being rigid, and chaotic. The paper also 
states that inside a CAS, patterns do develop, but they involve constant change, and that 
the patterns are formed by the behaviors of the “actors” in the CAS who affect each 
other, and the system as a whole [Ref 41]. Professor Jacques Ferber supports these 
behaviors in his book Multi Agent Systems [Ref 42]. 

In the case of the military personnel retention system, the components are the 
service members within the military (or within a subset of the military), and the group 
behavior is the extent to which the service members decide to stay in the military or get 
out of 1t (including how much service members affect each other). At times in our past, 
retention was very predictable; over the last ten years, 1t has fluctuated, at times seeming 
most chaotic, specifically in regard to our technical MOS and pilots. 

While the concept of CAS may seem foreign at first, we actually have a great deal 
of CAS around us in our daily lives. Some are natural, such as the cells in our body, and 
insect colonies; others are man-made (such as large-scale communications networks and 
artificial neural networks). Of these, insect colonies seem to be the easiest to understand, 
and perhaps the best example 1s that of an ant colony. 

Each ant colony is comprised of thousands of ants which go about their daily lives 
with little knowledge about the colony as a whole. The ants perform their duties, 
interacting with each other, and influencing each other. Over time, the colony exerts 
numerous colony-level behaviors, none of which were directed by a "commander" ant; 
the behaviors are simply the result of the mass interactions of the member ants. 

While there are numerous government agencies and private corporations studying 
CAS, the current research leader is the Santa Fe Institute (SFI), a private, non-profit, 
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multidisciplinary research and education center located in Santa Fe, New Mexico [Ref 
39]. One of the most significant CAS projects at SFI is the Swarm Development Group 
(SDG), a group that uses its own Swarm Simulation System (SSS) to pursue state-of-the- 
art multi-agent based simulations [Ref 40]. 

3. The Appropriate Model for a CAS 

As stated above, a CAS consists of up to millions of components or subsystems, 


> 


which are called “actors.” These actors all play similar, but different roles from one 
another. Worker bees have variations in how well, and how hard they do their work, and 
interact with each other; so do service members. In traditional models, we have to treat 
the individuals homogeneously; basically, we make them all the same. However, the 
aforementioned advances in technology and modern concepts allow us to create millions 
of unique individuals; the question now is how many different parameters to give the 
individuals! 

The concept that allows us to model down to the actor level is agent-based 
simulations. A computerized object called an “agent” represents each actor. Each agent 
has a set of variables, and functions that set, get, and change the values contained in the 
variables. The parameters are set by random number generators owned by higher-level 
(environment-level) programs, which ensure that duplicate agents are the exception, and 


not the rule. The system as a whole is termed a Multi-Agent System (MAS), and will be 


discussed in further detail in later sections. 
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р. SUMMARY 

Military personnel R&R behaviors are undergoing an unprecedented period of 
constant change. Modeling R&R behaviors thus becomes increasingly difficult as the 
behaviors depart from the constancy of the past to the almost chaotic behaviors they are 
exerting in today’s military environment. Our failure to properly understand these 
changes has had a significant impact upon the readiness of the Armed Forces. Modern 
computer technologies now allow for the generation of more complex computer models 
to simulate the often chaotic-appearing behaviors of military personnel specialties. These 
behaviors can best be identified as being those of a CAS, and can now be modeled by 
advanced computer programs. Examples of simulation models replicating CAS can be 


found at a number of research facilities and private corporations, such as the SFI. 
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Ш. PREVIOUS RESEARCH AND LITERATURE 


“Multi-agent systems bring a radically new solution to the very 
concept of modeling and simulation in environmental sciences, by offering 
the possibility of directly representing individuals, their behavior, and 
their interactions. Multi-agent simulation is based on the idea that it is 
possible to represent in computerized form the behavior of entities that are 
active in the world, and that it is thus possible to represent a phenomenon 
as the fruit of the interactions of an assembly of agents with their own 
operational autonomy.” --- Jacques Ferber 


This Thesis combines the studies of Military Personnel Retention, and modeling 
CAS with MAS. As the study of military retention has existed almost as long as the 
military, there are many sources for its study. The Naval Postgraduate School has led the 
way 1n this research, and has had numerous theses and papers published on the subject. 


This thesis will utilize several of the most recent and applicable NPS studies. 


The remainder of the background research for this thesis concerns modeling CAS. 
Unlike military retention, the study of CAS is relatively new. The concepts were 
spawned at the SFI back in the mid-1980's, but the computer power necessary to properly 
execute the concepts was only available by using supercomputers. As the speed of 
personal computers began to skyrocket during the 1990's, CAS modelers began to find 
the capacity to experiment more with the concept of computerized agents. As processor 
speeds have continued to increase, so has the use, and therefore study of CAS. П 15 


believed that this area will continue to expand and grow for the foreseeable future. 
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To date, only one paper has been found that combined the use of CAS and 
military retention (written at the Air Force Institute of Technology (AFIT) by Captain 
Marty Gaupp). Major Raymond Hill was Captain Gaupp’s thesis advisor, and is the 
AFIT point of contact for CAS research [Ref 48]. 


A. RETENTION RESEARCH 


Research on military retention 1s conducted throughout the Department of 
Defense. Materials for this thesis were located at the Naval Postgraduate School, the Air 
Force Institute of Technology, the Military Operations Research Society, and the Defense 
Management Data Center (DMDC). Additional information was provided by sources 
from within the Navy’s Bureau of Personnel Affairs (BUPERS), and the Army’s 
Personnel Command (PERSCOM). 

1; Тигпег 

In March 1995, Russell Turner published his NPS thesis The Impact of the 
Military Drawdown on USN Aviator Retention Rates. Tumer's Thesis was on how the 
drawdown affected the retention decisions of Naval Aviators. It was a timely thesis, in 
that it was written long enough after the beginning of the drawdown to analysis the 
effects, but was also soon enough to provide timely feedback for decision makers. Turner 
was one of the first military researchers to state that the military should focus on group 
behaviors instead of individual behaviors. 

Turner created a linear model in which he hoped the affects of the drawdown 
could be evaluated and then eliminated, thus leaving data that would have represented 
drawdown years that could be compared and contrasted with previous (and future) non- 


drawdown penods. Tumer cut out involuntary separations so that he ended up with data 
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only on those officers who made their own decisions. He then conducted a regression 
analysis on his data to provide a quantitative estimate that could provide personnel 
decision-makers with a tool to adjust bonus monies based upon the desired retention rates 
of specific aviator communities. 

Turner evaluated six factors in his study: the Aviation Continuation Pay (ACP), 
the Voluntary Separation Incentive (VSI), the Involuntary Reduction in Active Duty 
(RAD), Minimum Service Requirement 2 (MSR2), MSR3, and civilian unemployment. 
One significant area Turner failed to analyze was the civilian pilot hiring rates. 

Turner conducted a goodness-of-fit test on his factors and found that they 
provided a significant improvement in explaining the variation in Navy pilot continuation 
rates (CR). Turner’s conclusion stated that a one percent increase in bonus money will 
have such affects as a 15.7 percent increase in the CR of jet pilots, or a 22.2 percent 
increase in the CR of jet non-flying officers. 

Unfortunately, Turner's conclusions have not survived the test of time (which may 
have been drastically affected by the continued economic boom). Since the time of 
Tumer’s study, Navy pilot monetary incentive programs have continued to fail regardless 
of the additional monies they offered. To date, the Navy is still trying to find the nght 
amounts of bonus money to provide to each aviation communities in order to raise pilot 
retention [Ref 44]. 

2 ВооКһейпег 

William R. Bookheimer was also a student at the NPS. In March 1996, he 
published his thesis entitled Predicting Naval Aviator Attrition Using Economic Data 
[Ref 46]. 
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Bookheimer's Thesis was on predicting Naval pilot attrition using economic data. 
While it provides good information on monetary issues, 1t sidesteps other critical issues 
such as quality of life. Bookheimer's focus was clearly on what economic conditions 
could signal a change in the retention behaviors of pilots. His study utilizes data from 
1978 to 1990, which is now believed to be uncharacteristic of the current economy (with 
both the beginning and ending periods being in recessions). 

Bookheimer separated his studied group into six sub-populations by dividing 
them into their distinct aviation communities (jet, helicopter, and propeller) and by how 
many years of service they had (5-8 or 9-12). He did not use any pilots who had four 
years of less service due to their not being eligible to voluntarily separate. 

Bookheimer created three different linear regression models and found that no 
model was best at being able to predict retention rates for all six groups. He found that 
the national unemployment rate was the most useful factor that he studied, with the 
previous year's separation rate as the second most useful factor. He also found that the 
two groups that were most likely to separate were jet and propeller pilots with 5-8 years 
of service. These findings indicate that the Navy's biggest retention problem is at the 
pilot's first chance to separate. 

3: Gjurich 

Gregory D. Gjurich is a recent graduate of the NPS, having done so in March of 
1999. His thesis was entitled 4 Predictive Model of Surface Warfare Officer Retention: 
Factors Affecting Turnover [Ref 47]. 

Gjurich studied the factors that affect personnel turnover. Although his focus was 
on Naval Surface Warfare Officers (SWOs), his study shows many factors that affect all 
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specialties, particularly commissioned officers. Gjunch utilized logistic regression, and 
classification trees to determine how strongly these factors affect retention behavior. It is 
Gjurich's belief that these two methods are superior to the Navy's current process of 
extrapolating historical trends. He utilized data from over five thousand Officer Master 
Files (OMFs). Each officer was a Navy Lieutenant (officer pay grade 3/O3) that had 
already completed his/her initial obligation. 

Gjurich found that how well service members perceive their families are being 
taken care of had the strongest effect upon their retention decision. He also found that 
ROTC officers were more likely to stay in the service than Naval Academy officers, and 
that officers with any postgraduate schooling were more prone to stay in the Navy 
(showing that investing into the personal education of Naval Officers pays off). 

One of Gjurich's more interesting points that he made was that the Navy should 
not be interested in the retention of individuals, but rather should focus on groups of 
officers; this belief is in line with that of Complex Adaptive Systems (that group behavior 
is what we are trying to model and understand). 

4. Сапрр 

Martin P. Gaupp graduated simultaneously with Greg Gjurich (in March 1999), 
but from the Air Force Institute of Technology (AFIT), which is the major postgraduate 
institution for the Air Force. It is unfortunate that all four services do not use the same 
center for their postgraduate course because Gaupp apparently did not know of the 
retention research being conducted at the NPS by the three previous authors (two of 


which were aviation specific and therefore applicable to Gaupp's research). Аз Ше 
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Defense Technical Information Center’s (DTIC’s) database grows and improves, it may 
alleviate some of these problems. 

I found out about Gaupp’s research while visiting the AFIT’s website, and was 
able to obtain his thesis and additional materials from Gaupp’s thesis advisor, Major 
Raymond Hill. Since that time, NPS students have been asked to put AFIT on their 
distnbution for all agent-based and retention-based theses. It should also be noted that an 
AFIT doctorate student (Major Crino) is now considering furthering Gaupp’s work. 

Gaupp was the first military officer to utilize CAS to study military personnel 
retention. His thesis is entitled Pilot Inventory Complex Adaptive System (PICAS): An 
Artificial Life Approach to Managing Pilot Retention. When Gaupp started his thesis, the 
focus of Air Force retention was on pilot retention, as it has been for most of the history 
of the Air Force. Since October 1998, the Air Force has been expanded its retention 
concerns to many enlisted specialties [Ref 34, and Ref 49]. 

Gaupp’s model is very specific, developed for one special skill within one branch 
of the Armed Forces (Air Force Pilots). His goal was to develop an environment that 
captured the essential information present in the real world (that of Air Force pilots), and 
then to see what his agent pilots would do retention-wise. The model is limited in how 
many factors are utilized. The behaviors of pilots are controlled only by how much 
money they make, and how much time they get off from duty. The environment itself is 
created by only three factors; those being commercial airline pilot vacancies, the 
perceived civilian-military pay gap, and the flying OPSTEMPO. Gaupp provides a Java 
Applet that is utilized to provide the user a graphic user interface (GUI), where the user 


can manipulate the environmental factors. Gaupp created three basins of attractions, 
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based upon Chaos Theory (separate, retain, and undecided). It is Gaupp's contention that 
chaos theory plays a large role in-group behavior. 

Gaupp’s thesis has the significant challenge that all such theses will have: How 
do you display what is happening in an un-situated environment. In other words, how do 
you display something that normally isn't observable? In Roddy and Dickson's October 
2000 thesis, they provide an example of how to depict the El Farol Problem (agents 
choosing whether or not to go to a bar), and Unrath's October 2000 thesis also showed 
how agents can depict armed helicopter reconnaissance; however, these are situated 
environments, where one could actually observe these events occurring by standing 
outside the bar, or sitting inside the helicopter. In the case of un-situated models, the 
challenge is to try to show users what is going on. Gaupp chose to model his 
environment after Anel Dolan’s “Artificial Life on the Web, Java A-Life Experiments 


and Artist 3D Dolls.” Please see figure 3.1 for an example of Dolan’s A-Life model. 





Start | Continue s ] m. Е Slower | Lee | Default | Strange r | 


Figure 3.1: | Example of Ariel Dolan’s Artificial Life Display (from Ref 50) 
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э. H. E. Mills 

Mills studied the difference between two Navy pilot retention programs. Up until 
1998, the Navy utilized the Aviation Continuation Pay (ACP) bonus program, which 
provided bonus monies to target aviator groups. As long as an aviator stayed in he 
received that bonus. In 1998, the Navy switched to the Aviation Career Continuation Pay 
(ACCP) program, which tied the bonus monies to specific career positions. As long as 
aviators continued to advance their careers, they would receive the bonus money. 
Although the amount of ACCP data available to Mills was limited, his findings are 
encouraging. Mills found that the new program was causing positive impact upon pilots' 
likeliness to stay in the Navy, particularly in the crucial years of service (YOS) 11 — 20, 
which showed an increased likelihood of almost 20 percent. These results indicate that 
bonus money alone many not gain desired results; we must tie it to events and/or 
positions. 

6. Military Operations Research Society (MORS) 

The society 1s seen as the professional organization for military operations 


researchers. It maintains a website at www.mors.org. Its last symposium was held in 





June 1999 at the United States Military Academy (USMA). During this symposium, 
Working Group 20 (Manpower and Personnel) discussed a number of significant military 
personnel issues, to include retention. One of the more significant retention papers 


presented was that of CPT Marty Gaupp’s PICAS. 
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T William Krebs 

Professor William Krebs has done extensive analysis for the Department of 
Defense over the last decade in the area of forecasting the retention of Naval Aviators. In 
addition, he served as the thesis advisor for four of the above-mentioned authors (Mills, 
Gjurich, Bookheimer, and Turner). In 1998, Professor Krebs, along with Professor 
Samuel Buttrey, and several of their graduate. students conducted a retention attitude 
analysis upon approximately 1700 M and Marine Corps pilots [Ref 52]. This research 
found that aviators are best motivated by high-level needs such as co-worker satisfaction, 
and job satisfaction. They also found that pay satisfaction was more important for senior 
officers than junior officers. They further found that retirement pay was significantly 
more important to retention than was bonus pay (ACP/ACCP), though it should be noted 
that this study was conducted prior to the repealing of the Redux pay program. 

Aviator retention is as significant a problem for the Navy as it is to the Air Force. 
Figure 3.2 below reflects the retention likelihood found by Professors Krebs and Buttrey 
during their survey. This survey indicates that all aviator groups have significant 
retention challenges, with the sole exception of non-flying officers (NFO’s) with more 
than 14 YOS; all other groups have at least 40 percent likelihood of voluntary separation 


prior to retirement. 


59 


Are you considering leaving the military? 
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— Navy Pilot 
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Figure 3.2: | Pilots who planned to depart the Navy in 1998 (from Ref 52) 
B. COMPLEX ADAPTIVE SYSTEMS RESEARCH 

1. John Holland 

John Holland is seen by scientists as the father of Complex Adaptive Systems 
(CAS). He has written several books that discuss the development of Genetic Algorithms 
(GA) and CAS. In his book Hidden Order — How Adaptation Builds Complexity [Ref 4], 
Holland outlines principles for both CAS and GA, and also demonstrates many of the 
procedures and approaches for them. These approaches are now being used to study 
systems in a vanety of sciences, to include biological, social, and environmental sciences. 

In Hidden Order, Holland established a framework for creating MAS that model 
CAS (based on a cellular schema), which he called Echo. Unfortunately, Echo was built 
on a now outdated version of C-code. A newer, Java-based, relational architecture has 
just been released by Roddy and Dickerson, which should be seen as a must-read for all 


those seeking to create new CAS simulations [Ref 22]. 
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Hidden Order was onginally presented as a lecture series at the SFI in 1994. 
Holland’s follow-on book was released in September 1999, and entitled Emergence — 
From Chaos to Order [Ref 5]. In this book, Holland provides further insight into CAS, 
particularly focusing on emergent behaviors of systems. An underlining theme of this 
latest book 1s that complex systems as a whole are more than just the sum of their 
individual parts. 

D. Richard Dawkins 

Richard Dawkins is an Oxford-educated author of numerous books about 
evolution and science. In his book River out of Eden — A Darwinian View of Life [Ref 6], 
he provides outstanding insight for anyone trying to create an agent-based system that 
will evolve and change over time. Although the book appears to be primarily aimed at 
biology, readers with an understanding of Holland’s GA’s can see how it applies to CAS. 
Dawkins’ book does a good job of describing how intra-species natural selection achieves 
a self-organizing structure. This book describes how organisms evolve over long periods 
of time (gradualism) rather than through creationism or the leaps and bounds that many 
scientists often imply in their works. Dawkins states that the organisms interact with 
each other, and thus form a system that itself evolves over time. 

An interesting point that Dawkins makes in this book is about natural selection. 
Dawkins rightfully points out that it isn’t so much the survival of the specific genes, but 
the survival of the organism that contains the genes. There may very well have been 
genes that may have given humans far superior skills over what we possess today --- if 
only their carrier had managed to procreate before he/she died! This leads us to 
understand that individuals within CAS mutate from time to time; therefore the MAS that 
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model a CAS should contain a certain amount of random vanation (thus modeling 
mutations). 

3. Mitchell Resnick 

Resnick works for the Media Laboratory at the Massachusetts Institute of 
Technology (MIT). His 1997 book entitled Turtles, Termites, and Traffic Jams: 
Explorations in Massively Parallel Microworlds helps to show why Complex Adaptive 
Systems (CAS) may be the best way to study systems with many moving parts [Ref 7]. 
In this book, Resnick provides guidance on massive parallelism (the simultaneous 
processing of large amounts of adaptive agents), which 1s necessary for running MAS. 

Resnick uses conceptual turtles and insects as teaching tools in order to help 
describe the principles of parallel processing and system  self-organization 
(decentralization). He also attempts to explain why the CAS modeling movement is 
being resisted (by people having problems understanding decentralization concepts such 
as bird flock leadership — the flocks have no central leader, but rather each bird takes a 
turn being the lead bird in order to share the work load). “Termites” provides free 
software for simulating self-organizing system behaviors (StarLogo, based upon Logo, 
which can be found at the website: www.media.mit.edu/groups/el/projects/starlogo/). 

4. Andrew Пасһіп5Кі 

Andrew llachinski is the militarys leading researcher in Complex Adaptive 
Systems. His ISAAC model was the first CAS model designed for modeling land 
warfare (designed for the Marine Corps). llachinski has shown that the collective 
decentralized interactions among individual agents obeying local rules can often appear 
disordered at their individual level, but may actually be creating a higher-level pattern of 


42 


behavior. Previous land-based war models all used Lanchester equations to calculate 
losses; the military's field manuals are full of rates that have been developed using these 
equations (e.g. FM 101-10-1). However, there are many limitations to using these 
equations for modeling modern combat. Of particular note, they do not take human 
factors into consideration. Many battles have been won or lost by the beliefs of 
individuals that spread, and developed into group behaviors that are not predictable by 
linear equations. 

The Civil War battle of Chickamauga is a good example of group behaviors 
overcoming linear logic. Despite the Confederate Army flanking the Union Army, a 
small group of individuals, led by General George H. Thomas, created a group dynamic 
which caused the turn of the battle from a Union defeat, into one of its most important 
victories of the War. Modern warfare takes this dynamic a further step; warfare is no 
longer properly characterized by large, homogeneous forces. Today's warfare is better 
characterized by relatively small groups of highly trained individuals, interacting with an 
enemy in a constantly changing environment. The future for military operations appears 
to be continuous OOTW. 

ISAAC is set up in an environment that contains the following characteristics that 
should be further studied for situated environments: A default local-rule set for 
individual agents; goal directed behavior; sensors that generate an internal map of the 
environment (a situational awareness) for the individual; an internal adaptive mechanism 
to alter behaviors. An example of Dr. Ilachinski’s ISAAC can be found at figure 3.3. 

The ISAAC simulation is a situated environment, playing a version of the 
“capture the flag” game; two armies compete against each other, with the ultimate army 
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goal being capturing the enemy flag. Unfortunately, the agents inside of the armies do 
not always weigh capturing the flag as being more important than surviving. Thus 
ISAAC is an example of agents that are more than just objects; they may or may not do 
what they are told (just like real Marines and Soldiers). 

ISAAC has an outstanding movement algorithm called the “least penalty 
function” (LPF), which dictates where agents move on the simulated battlefield. . The 
LPF identifies each possible location where an agent can move, and what is the least 
painful place to go. If an agent weighs survival more than capturing the flag, then it will 
not risk its life in order to capture it. 

Students at the NPS (under the guidance of Professor John Hiles) created their 
own versions of the ISAAC simulation, putting focus onto meta-agents (agents with 
higher level roles such as leadership). The students created local commanders (squad 
level leaders), and global commanders (leadership who direct junior leaders). The 
commanders can be played by the user, and have much more information about the battle 
than the basic agents. In these simulations, agents have to decide on whether or not to 
follow orders given to them by their commanders. An example of the student-created 
models (created by Major Jason Stine with the assistance of Major Stevan French, Major 
Mark Tanner, and Major David LaFlam) can be found at figure 3.4. 

A challenge for ISAAC-based simulations is to represent perpetual knowledge 
denved after the simulation is over. In ISAAC, each simulation stands alone, so the 
survivors from one simulation do not pass on what they learned to those that conduct the 
next simulation. However, in a military organization, service members hold after action 
reports (AAR), in order to leam from their combined expenences. As the members of the 
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organization rotate out (and new ones come in) the organization still maintains a great 
deal of the knowledge acquired (organizational knowledge). Thus the unit does not have 
to re-experience every task again to know what is the best way to complete a mission in a 


given scenario. 
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Figure 3.3: Example of Andrew Ilachinski’s ISAAC simulation (from Ref 8) 
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Figure 3.4: Example of NPS Student version of ISAAC Combat Simulation (Stine, 
French, Tanner, LaFlam) 
5. Eric Bonabeau, Guy Theraulaz, and Marco Dorigo 
In their book “From Natural to Artificial Systems,” the authors detail their studies 
on the collective behaviors of social insects, and uses the studies to show how to created 
distributed algorithms, and multi-agent systems in order to derive group behaviors [Ref 
11]. Like Resnick, these authors see the decentralized behaviors of social insects as the 


most appropnate model for evaluating and imitating group behaviors. The authors 
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believe that social insects have three key features, decentralization, flexibility, and 
robustness. They define "decentralization" as meaning that decision-making is conducted 
at the lowest possible level. They define “flexibility” as meaning that adaptation occurs 
in the environment, but the system continues. They further define “robustness” as the 
fact that no matter how well or poorly an individual may do its job, the insect colony 
continues (the failure of one component of the system does not necessarily cause the 
whole system to fail). 

The authors believe that as the world becomes more complex, the top-down 
simulations we use to model it become increasingly ineffective. However, regardless of 
how complicated the world becomes, the individuals in it are still the same basic building 
blocks for society. The authors term the phrase "swarm intelligence” to indicate a system 
where "autonomy, emergence and distributed functioning replace control, 
preprogramming and centralization.” 

6. James Gleick 

James Gleick is a science writer, who worked for The New York Times in 1987 
when he authored his book “Chaos: Making a New Science”. Gleick wrote the book to 
tell about the work of numerous scientists that were trying to find order in what was 
supposed to be total disorder: Chaos. As computers are able to process data faster, these 
scientists are able to analyze their data in more depth. Many of these scientists have been 
able to find complex patterns developing where they previously believed no logic could 
possibly prevail. 

Although Gleick did not know of things such as agents, CAS, or MAS, he did 
write about multi-disciplinary approaches towards studying systems, and the complex 
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behaviors that they emitted. Gleick points out that a common theme about the scientist 
studying chaotic systems is their ability to identify complex patterns where others 
initially saw nothing. Figure 3.5 shows just one of the many patterns that these scientist 


have found from chaos (this one was a weather pattern found by Dr. Edward Lonrenz). 





Figure 3.5: Weather Pattern Found by Dr. Edward Lorenz, found in James Gleick’s 
Chaos (from Ref 12) 
= Robert Axelrod 
Axelrod ıs the author of several books on the topıc of complexity, to ınclude “The 
Evolution of Cooperation” [Ref 53] and “The Complexity of Cooperation” [Ref 54]. 
Axelrod writes about the socıal sciences, but he does so from the viewpoint of a computer 
scientist. "Complexity" contains variations of his most famous computer simulation (that 


of the story of “The Pnsoner’s Dilemma”). This agent-based simulation is a series of 
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identical scenarios between two individuals, each of which can “rat out” the other 
prisoner, or can keep quiet. If they both keep quiet, they both get a small punishment; if 
they both talk, they both get more punishment; if only one talks, he goes free, and the 
other gets a significant amount of punishment (the computer simulation scores the 
scenario slightly differently). Axelrod’s challenge to all programmers is to create a 
program that will excel at the dilemma over the long run, with random errors thrown in to 
what the other player is doing. Programs that are too forgiving or too vengeful all seem 
to lose; Axelrod has found over time (and after many competitions) that it is the agent- 
based programs that cooperate with each other (including anticipating the errors) that win 
the game. This includes allowing an opponent to strike back when we accidentally take 
advantage of them. 

8. Natalie Glance and Bernardo Huberman 

Glance and Huberman wrote an article for the March 1994 edition of Scientific 
American, in which they introduced a scenario similar to Axelrod’s Prisoner’s Dilemma, 
called the "Diners Dilemma." The article was entitled “The Dynamics of Social 
Dilemmas.” This simulation was more complex though, because there was more than 
two actors in the simulations, and not all of the actors new everything that was going on. 
The scenario entailed a group of friends that periodically go to lunch, and instead of 
getting separate bills, they agree to split the bill evenly each time. This scenario is 
perfect for agents, as some will try to be gracious, and not order expensive meals, while 
others will always order the most expensive item on the menu; linear programming 


would result in all actors playing the game in the same manner. 
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This simulation also introduces the concept of Metanorms (also used by Axelrod), 
where actors try to police the actions of other actors. One of the goals of this scenario is 
to create agent-actors that are able to maximize their gains without being policed. Each 
simulation always results in one of two states, either many people cooperate with each 
other, and the system succeeds, or most fail to cooperate, and the system fails. 

2 Jacques Ferber 

Ferber was the first person to ever attempt to explain multi-agent systems in a 
textbook format. His book entitled “Multi-Agent Systems: An Introduction to 
Distributed Artificial Intelligence” was translated from his native French into English, 
and published in November 1998. It was quickly picked up by professors at the Naval 
Postgraduate School as being the most concise book from which to teach MAS concepts. 
In the book, he explains MAS basic principles, organizations, system actions and 
behaviors, communications, collaborations, and coordination of actions. The most 
important definitions that Ferber provides are what an agent is, and what a MAS consists 
of. Ferber’s work serves as the backbone of this thesis, and will therefore serve as the 
source for most in-depth definitions provided in section D of this chapter. 

10. Swarm 

“Swarm” was originally created by Chris Langton (who would later form the 
Swarm Corporation of New Mexico). Over the years, Langton has had many people help 
in the continual development of Swarm, many of who worked for the Santa Fe 
Institute(SFI) at the time of their work. The best piece of current Swarm work was 


accomplished by Benedikt Stefansson, who created a tutorial on Swarm in 1999 while he 
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was at UCLA (that tutorial is still available on the internet at the SFI: 
www.santafe.edu/projects/swarm/swarmdocs/overbook/overbook.html). 

The Swarm tutorial discusses how complex adaptive systems can be created, and 
appears to be primarily aimed at modeling economics (as one of its goals is to unite those 
trying to create economic simulations). It describes the main contributions of a Swarn 
concept as being any of the following: Event Management, Information Management, 
Graphical User Interfaces (GUIs), Memory Management, and the ability to support 
multiple software languages. It provides an overview of each of these uses. The briefing 
also provides an overview of Object Oriented Programming (OOP) for those who may 
not have had experience with this concept (which allows us to create proper 
environments for our agents). 

Stefansson has just recently furthered his work by publishing a book on Swarm 
with Fracesco Luna, entitled “Economic Simulations in Swarm: Agent-Based Modeling 
and Object Oriented Programming” [Ref 56]. More information about Swarm can be 
found at their official website: www.swarm.ore/index.html. More information about SF] 
can be found at: www.santafe.edu. 

11. Craig Reynolds 

In 1986 Craig Reynolds created the first of his computer models that simulated 
the flocking behaviors of birds (which he called “Boids”). The model serves as a prime 
example of how simple rules can often describe what appears to be a complicated system. 
It also shows that models do not have to be centralized (driven from the top-down), but 


can often run better and smoother when control is decentralized. 


Si 


Reynold’s initial model utilized three rules: separation (don’t get too close), 
alignment (go in the general direction of all nearby boids), and cohesion (don’t get too far 
away from nearby boids). Each boid 1s an agent, and runs the set of rules for itself; thus 
the decisions made by one boid may be entirely different than the decisions made by 
other boids. 

Reynolds has continued to improve his model, to include adding obstacle 
avoidance. The current model has the birds on the ground, reacting to vehicles driving by 
them; if a vehicle gets too close, they take a small flight to avoid it; as they continue to 
get harassed by vehicles, they take longer flights, and cause more and more of their 
neighbors to react. Reynolds models would eventually be used as the basis for animated 
group animal behaviors in movies such as the bat swarms in Batman Returns, and the 
wildebeest stampede in Zhe Lion King. His work on boids can be found at the website: 
www.red.com/cwr/boids.html. 

12: Georgia Tech University Animation Lab: Group Behaviors 

Georgia Tech has some wonderful demonstrations of agent behaviors captured on 
MPEG’s. From bicyclist avoiding road hazards, to one-legged robots hoping around 
objects, observers are able to see that the individual decisions create group behaviors. 
‚html. 


The website is: www.cc.eatech.edu/evu/animation/Areas/eroup behavior/erou 





15: Brian Arthur 

Brian Arthur created the El Farol simulation, which is a great tool for both 
explaining/analyzing agent behaviors, and for teaching beginning agent-based 
programmers. The basic simulation replicates the behaviors of 100 individuals who all 


enjoy going to the El Farol Bar in Santa Fe, New Mexico on Tuesday nights. Like most 
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bars, ıf ıt gets too crowded, people do not have a good time; in this case, 60 or more 
people constitute too crowded. Each patron-agent decides whether or not to go to the bar 
based upon rules that they are given when they are created (which are different for each 
agent), and their past experiences of going to the bar. 

Each agent makes its decision based upon its heaviest weighted rule (which 1s 
called the primary rule ); however, all rules are reinforced based upon what would have 
happened if they had been that primary rule. Over time, poor rules will be discarded, and 
new rules will be added. Although the agents do not get to talk to each other, at times 
many agents will utilize the same primary rule. The most interesting phenomena that 
come out of the El Farol simulation 1s that the number of agents that actually attend the 
bar begins to develop a pattern (indicating that the system 1s not chaotic, but merely 
operating near chaos). 

An example of output from an NPS student version (Major Mark Tanner and 
Major Stevan French) of the El Farol Problem 1s found at figure 3.6. The figure appears 
to start in a state of chaos, until approximately week (simulation) 350, when 1t appears to 
be forming a static pattern. This pattern maintains for about 60 weeks, until the 
simulation slips away from this pattern. 

When Major Tanner and I originally began analysis of our El Farol simulations, 
we stopped at 500 weeks, and were somewhat happy with the patterns we saw. However, 
we soon began asking ourselves why we should stop at 500 weeks when the simulation 
did not take a great deal of processing time. We therefore decided to expand the 
simulation to 2000 weeks and found our results to be more impressive. Figure 3.6 1s just 


one of the many examples we found of the system creating what appeared to be group- 


33 


wise decisions about which agents would attend on which weeks. The last 1000+ weeks 
of this figure reflect a pattern of 31 attendance numbers repeating over 30 times before it 


eventually went back into a non-static pattern. 





Figure 3. 


с. RELEVANT ARTICLES 


During my research, I found a number of articles that should be read by anyone 
studying agents and/or retention. As these articles will help the reader gain a better 
overall knowledge about these two areas, I have included synopses on the four articles 1 


have found to be most relevant. 
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1. An Exploration into Computer Games and Computer Generated 
Forces 

John Laird of the University of Michigan wrote this paper for the Eighth 
Conference on Computer Generated Forces and Behavior Representation [Ref 57]. Ш 
this paper, Laird compared and contrasted computer games (CG’s) and computer- 
generated forces (CGF’s). The synopsis is as follows: 

Laird wrote about how computer games and computer simulations have improved 
dramatically over the last five years. He attributes most of this improvement to the 
increase in available processing speed. This has allowed designers and/or programmers 
to make more complex environments and objects within those environments. Probably 
the most important objects within the programs are those that represent players. Inside 
computer games, we find what Laird terms Al's (which are the computer generated 
characters that users interact with). Inside computer simulations, we find what Laird 
terms CGF's (Computer Generated Forces). Both use avatars to represent these 
interactive objects; in most cases, these avatars are humans, but they can also be pieces of 
equipment such as ships and planes. 

Computer Simulations are the older of the two programming technologies, having 
been spawned by funding from the Department of Defense. During the Cold War, we 
needed bigger and better simulations to help with our war planning and preparation. Our 
government therefore utilized hundreds of millions of dollars into this area. Thus the 
research that went into CGF's far surpassed that of computer games until after the Cold 


War was won. 
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As the technological revolution took off, computer games began an exponential 
increase in usage and development. Al's therefore began to benefit from the commercial 
development. Although they still do not meet the standards of their CGF cousins, they 
are improving at a faster rate than that of Al's. It could very well be that Al's may 
eventually supersede CGF's in complexity and realism. 

Currently, Al's do have several advantages over CGF's. First, their designers and 
programmers are free to design whatever they want. The final product is whatever it is 
(designs are subject to change, and are much more flexible than CGF's). CGF designers 
have to make sure that their products closely replicate human behaviors, and have to 
worry about negative training effects of any interactions which are not human realistic. 
A second advantage is that designers are free to select and/or change their standards for 
their Al's, while CGF designs must follow ngid DoD guidelines. A final advantage is 
lower cost due to lower requirements. 

The main positive for CGF's is that they are scalable and expandable. CGF 
scenarios are written with few specifics, so that users can create their own scenarios. 
CGF's are created in mass before any scenario is ever played out. They are then selected, 
and inserted into a given scenario. On the other hand, AI's must be written to exact 
scenarios, and normally have to be redesigned from scratch whenever the situation 
changes. 

It should be noted that the goals of CGF's and Al's are orthogonal. The goal of 
CGF’s is to provide realistic training for their users; the focus 1s on accuracy compared to 
the real world. CGF's must behave like comparable humans would. CGF simulation 


may inadvertently be fun to interact with, but enjoyment is not a requirement. Al's are 
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the exact opposite; the goal is fun, fun, and only fun. With Al's, it is often better if the 
objects have superhuman powers (especially if the AI is representing the user). 

To meet these ends, the Al's are programmed differently than CGF's. While 
CGF's can only have the information that they have acquired, Al's often have global 
knowledge about the entire game! This is done so that each AI can best play its role in 
the scenario that is being run. This is all done for the sake of making the game more 
interesting for the user. This global knowledge allows the AI's to anticipate what the user 
15 going to do next. This would be next to impossible if the Al's only had local 
knowledge. 

CGF's and Al's fit into game modules differently. The logic that runs CGF's is 
normally in separate modules than the main program running the given scenario (Laird 
states that all major DoD simulations have this encapsulation, to include the simulation 
that he has developed with the Air Force, TacAir-Soar). AI’s on the other hand, are often 
mixed in with the game engine itself. This makes it difficult to separate the AI from the 
scenario, or the scenario from the game engine (kind of like the Army simulation 
converted into the kangaroo migration model; when a simulated helicopter flew over, a 
kangaroo pulled out a rocket launcher and shot down the helicopter!). 

The one thing that Laird appears to be focusing in on about APs is the 
predictability of most Al’s inside of computer games (think embodied, human-like 
avatars doing the same thing at the same point of every game you play). Laird himself is 
a computer game designer, and he wants to eliminate this predictability. The cost of 


predictability is eventual boredom. A game that is the same every time can be fun until it 
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is mastered, and then it is put on a shelf (or worse, into the garbage). Laird looks to the 
СОЕ 5 as a potential source for solving the problem of predictability. 

Development time for the two types of programs differs significantly, thus 
affecting the complexity of their avatars. Computer simulations normally have several 
years of development before being used; this allows for the development of complex 
CGF’s. Computer games normally only have 1 year of development time, and thus there 
is significantly less time for developing the behaviors of AI's. There appears to be a 
casual affect between development time and usage; the programs that take the most time 
to develop (computer simulations with their CGF's) normally get used longer. Short, 
quick, and quick programs (like computer games) take less time to develop, and then are 
discarded much faster. We should all note that some games (like the Sims) take longer 
to develop, and provide a higher quality, less predictable product; these games have been 
known to be used much longer than standard computer games. 

Another factor that affects the life cycles of APs and CGF's is that of reusability. 
Often times, the AI's are written into a specific game, and are interwoven like Siamese 
twins. [n such cases, programmers will often create the AI's again from scratch rather 
than try to separate an AI from one game in order to use it for another game. 

We should also note that AI's are normally only the creation of one person on a 
video game project team. This person 1s usually allocated only 5-10% of the CPU of the 
computer the game is played on (the rest going to the graphics, networking, sound, game 
play, physics etc). 

Al logic is not normally programmed into computer games in code that 
maximizes the capabilities of expert systems (like Prolog or Lisp). Instead, the logic 15 
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converted into the native language of the game (C, C++, Java) through conditional 
statements (“if’s”, etc.). Preprocessing allows for faster processing of the game. Laird 
feels that with the ever-increasing speeds of CPUs will usurp any graphics requirements, 
thus allowing more CPU cycles towards creating more accurate behavior models (AI’s). 
This in turn will allow more complex AT's to be developed. 

Laird feels that CGF research could benefit from computer games. First of all, 
many computer games provide much better detailed environments than computer 
simulations. Games like Doom are very capable of handling the massive numbers of 
Al's/CGF's that most simulations need. Second, many games create and provide DLLs 
which make it possible for networked events to occur, even when all users do not have 
the source code (a great potential in cost savings). Third, most games are easy to learn, 
and users are utilizing them almost as quickly as they can open the boxes that the games 
come in! 

In order to better understand computer games, Laird created a simulations that he 
emplaced into the computer game "Quake IL" he called his AI the Soar *Quakebot." The 
Soar Quakebot is run on a separate CPU, and interacts with the game using the Quake II 
interface DLL. Laird uses Socket ИО in order to provide a platform-independent 
mechanism for transmitting all perception and motor information between the Quakebot 
and the game. 

The Quakebot uses the AI engine that Laird created (Soar). All ofthe information 
that the Quakebot needs to play the game is stored inside Soar rules. The game engine 1s 


able to interact with Soar up to 10 times a second (for updating information about the 
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environment and the Quakebot itself). Soar takes between 5-10% of the CPU cycles 
from its host (400MHz Pentium II running Windows NT). 

The Soar Quakebot utilizes principles that Laird learned from other simulations in 
which he needed to control entities (the latest being a combat pilot simulator). The main 
purpose of the Soar engine is to make and execute decisions. It utilizes operators 
(primitive actions, internal actions, or abstract goals) to drive the decisions it makes (all 
are eventually broken down into primitive actions taken one by one). Final executions 
are implemented by if-then rules. The Quakebot's long-term knowledge is stored in 
Rules. The Quakebot also has a working memory that it uses for the current situation. 
All of this works fine for reacting to current situations. 

But how do you deal with what “might happen?” This calls for anticipation. 
Traditional rule-based systems would say that you should write a rule for every possible 
situation (like Deep Blue); thus you would have an engine with every possible state 
(finite-state machine). All you need to do is write in all of the rules. But what about if 
there are so many rules to process, that you enemy acts before you can anticipate 15 
actions (then you are DEAD!). 

Unfortunately, life, and first person shooter (FPS) games do not have a set limit of 
possible situations. I do not have to move forward, I could simply choose to go 
backward, or even stay in place! In the case of the Quakebot, it runs on normal rules 
until it senses an enemy out of weapons range. It then runs an internal program about the 
enemy in order to determine what it would do if it were the enemy. This program is only 
run when it senses the enemy, but the enemy cannot be engaged at that time. It predicts 


what the enemy will do, and then takes its own actions to counter that movement. 
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After anticipating what the enemy will do, the Quakebot makes a decision on 
what 1t will do based upon one of three operators. It simultaneously wants to hunt, 
ambush, and deny powerups from its enemy. After analyzing the situation, the Quakebot 
will make the decision that best benefits it. After a given scenario, the Quakebot will 
"chunk" the situation into a preprocessed decision should the same situation arise again 
(thus it has completed a form of learning). 

The anticipation model can get wrapped around itself. What if the enemy is 
another Quakebot? Shouldn’t the first Quakebot anticipate that the other Quakebot is 
anticipating ıt? (Recursion!!!) In Quake, the problem normally doesn't arise, as the first 
bot to sense the other normally attacks and they both end up simply shooting at each 
other until one is killed. Another interesting scenario is that not all enemies act alike. 
Quakebots have got to anticipate that the strategies employed by one type of person may 
not be the same as others (just like different types of humans fight differently, and lesson 
we leamed all too well in Vietnam). The Quakebot will have to figure out what type of 
enemy it 1s facing before it tries to figure out what it will do. The Quakebot model is 
interesting, but obviously needs to deal with a dynamic situation versus the static one that 
it 1s designed to face. 

Throughout the paper, Laird talks all around Multi-Agent Systems, but he never 
uses the term. Laird also avoids using the term “agent,” instead using the abbreviation 
“АТ,” which is not defined in the paper nor on his website (though he does use Al and 


agent interchangeably on his website). 
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2: Pedagogical Agents 

W. Lewis Johnson wrote this paper for the Center for Advanced Research in 
Technology for Education (CARTE), University of Southern California/Information 
Sciences Institute [Ref 58]. Johnson was a guest speaker at the Agents 2000 Annual 
Conference held in Barcelona Spain (and is the Treasurer for the 5" Annual Conference 
to be held in Montreal, Canada in 2001), and is seen as one of the leading minds in agent 
research. The synopsis of Johnson's article is as follows: 

Autonomous Agents are becoming well known because of their abilities to mimic 
human behavior. This capability is now being extended in order to train humans in 
Virtual Environments (VE's). These special types of agents are called “Pedagogical 
Agents” (PA). These agents are a step above current computer-assisted tutoring systems. 
PA’s go beyond by interacting with students in several different, yet simultaneous modes 
(multi-modal). These agents are also able to provide feedback to students during training 
that seems similar to what a student would expect from a human tutor. 

One of the best parts of the PA’s is that they can be made to look similar to 
humans, to include gestures and facial expressions. This allows for enhanced 
communications with humans. This behavior has to be appropriate for the situation at 
hand, and has to be supported by accurate information being provided by the PA. 

PA development has been a science of the 1990's. Part of the focus has been on 
developing computer programs that act as peers instead of all-knowing instructors. The 
PA's help versus instruct. PA's must be written to anticipate any type of behaviors from 
the students or other PA's they may be working with. As humans can be irrational, and 
do things unexpectedly, the PA's must be prepared to react to just about anything. 
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Furthermore, the students that they tutor could have anywhere from novice to expert 
knowledge on the subject at hand; the PA’s must be flexible enough to teach at any level. 
For all of these reasons, PA programs must be robust. 

PA”s are not just step-by-step instructors. They must be prepared to provide hints 
on what to do next, or to answer the myriad of “what if” questions that a 3-vear old might 
ask a parent. The PA must therefore have a deeper knowledge of what knowledge 
encompasses the subject it 1s tutoring. 

Ihe article provides information on several PA’s. The first is CARTES' own 
PA’s, STEVE (Soar Training Expert for Virtual Environments) and ADELE (Agent for 
Distance Learning — Light Edition). Steve is the PA that best represents what many 
simulations students are currently studying: Training in immersed virtual environments 
(VE). STEVE an advanced prototype designed to interact with students in networked 
immersive virtual environments. STEVE’s initial tasks included working in VE’s that 


simulated US Navy surface ships. An example of STEVE is found at figure 3.7 below. 





Figure 3.7: STEVE (in blue) TX engineering аа to 2 АСК (from Ref 57) 


ADELE is a PA designed for desktop VE’s. She 1s like a high-speed version of 
the Microsoft Office Assistant, but with a special task in mind: Instructing medical 
students. ADELE presents medical situations to students. and allows the students to 
decide what the proper treatment is for the patient.. If the student makes a bad decision, 
or needs assistance, ADELE interacts with the student. See figure 3.8 for an example of 


Adele, and figure 3.9 for an example of the many current forms of the Office Assistant. 
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Figure 3.8: Adele offering advice to a medical student (from Ref 57) 
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Figure 3.9: Тһе 8 forms of the agent Microsoft Assistant (from MS Office 2000) 
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There are several other PA”s that have been developed. North Carolina State 
University’s Multimedia Laboratory has developed two pedagogical agents: “Herman the 
Bug,” and Cosmo. Herman”s purpose is to teach children about plants. Cosmo would be 
a big assistance for teaching classes on computer networks; Cosmo helps to work out 
computer network problems. The University of Saarbrucken has also developed a PA 
that 1s a super-Paper Clip, called “PPP Persona.” This PA helps students get through 
web-based tutorials in order to enhance their leaming and hopefully answer questions that 
they might have (similar to what a real professor would do in a real class): Thus making 
professors expendable! 

Interactions between PA’s and human students can take the full gambit between 
Microsoft’s Office Assistant and STEVE. In order to determine which is best for a given 
situation, one must analyze the task to determine just how much interaction is needed. If 
only hints are needed, then the Paper Clip method is probably best; however, if there are 
many interactions that should take place (or 1f demonstrations on how a human should 
execute a task are necessary), then it is best to have a one-on-one instructor such as 
STEVE: 

PA’s can also serve as extra humans during collective training. Should a squad of 
soldiers need to practice a task, STEVE PA’s could serve as any missing members of the 
squad. Thus there could be a squad that is half human, and half PA’s. If a baseball 
player wanted to practice skills at different positions, he could literally have 8 PA’s 
playing the other positions. The player could then move to different positions whenever 


he wanted to, with PA’s filling in each position that he left. 
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3. Embodiment in Conversational Interfaces: Rea 

Justine Cassell is the director of the Gestures and Narrative Language Research 
Group at MIT’s Media Laboratory. Her focus on Embodied Conversational Agents 
shows the extent to which an agent can be developed in order to interact with humans. A 
synopsis of her work, published for ACM Magazine [Ref 59] in May 2000 is as follows: 

Embodied Conversational Characters (ECC) are the logical extension of human — 
computer interaction (HCI) metaphor. Instead of keyboard input and text-screen output, 
(or even voice recognition and conversion programs), ECC provide HCI in the form of a 
two-way flowing conversation. ECC therefore allows for the nchness of human-like face- 
to-face communication, which goes way beyond command-response processing. This 
technology will eventually allow artificially intelligent tutoring systems to provide actual 
conversations with students, and real, independent feedback. In order for this human- 
computer conversation to work, the authors had to properly understand how it is that 
humans actually communicate with each other. Thus they conducted a Human-Human 
Conversational Analysis (HHCA). 

HHCA describes sets of conversational behaviors that fulfill conversational 
functions. Conversational functions were broken down into two components, 
Prepositional (verbal), and Interactional (non-verbal). Dialogues were also broken down 
into two categories, those being Social (greetings, etc.), and Task-onented (getting the 
task completed). 

The authors felt that the best mechanism for HCI was the use of human-like 
avatars, which they call Embodied Conversational Interface Agents. The authors have 
steadily improved upon their products over the last five years. This progression (in both 
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papers written, and software products produced) can be seen at the Web Site: 
gn.www.media.mit.edu/eroups/gn/. The first agents were robot-like Avatars that 
simulated human movements (this was called Body Chat, an example of which can be 
found at figure 3.10): Later generations made the characters more human-like, with 
examples being Eddie Edit and Gandalf (both not pictured), two programs designed to 


help children learn. 





So how are; | — 


Ready p NUM > 
(omi s P AP dh I PIN m о ЛАА. u TE A 


Figure 3.10: Embodied Conversational Agents (from Ref 59) 

The latest, greatest generation ıs a full-bodied autonomous agent created at MIT, 
called RE (which 1s short for Real Estate Agent (what “/she” emulates). REA has a fully 
articulated body, which she uses to send non-verbal cues/ messages to the user. Some 
examples of this include her turning toward approaching customers, waving and initiating 
conversations, her raising her hands when she wants to talk, her raising her eyebrows 


when she wants feedback, and her turning away from users when done. 


REA plavs the role of a real estate agent because the authors believed that this 
“job” would provide the necessarv range of challenges. Hence REA must interpret and 
provide both verbal and non-verbal communications, and interact in both a social and 


task-oriented manner. A picture of REA can be found at figure 3.11. 
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Figure 3.11: REA greeung -ж-: de Ref 59) 

Many other new software programs are capable of providing output in both verbal 
and non-verbal manners. A good example of this is Ananova, found at 
www.ananova.com. Ana provides a number of facial gestures while she "reads" the new 
through a text-to-voice interpreter. However, most of these programs do not solicit input. 
Of those that do, thev onlv input text (and only provide canned answers). A picture of 
Ana can be found at figure 3.12. 

REA is superior to these programs because she can input both verbal and non- 
verbal communications. Rea inputs active and passive inputs through manv devices such 
as user microphones for active input, and cameras aimed at the user for passive input. 


Rea combines all of these inputs in order to determine what the user 1s doing, and which 
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input is the most important. Rea actually synthesizes her responses (including speech and 
accompanying hand gestures) based on the current grammar, lexicon, and communicative 


context. 





Figure 3.12: Ana “reading” the news (from www.ananova.com) 

4. Twitch Speed, Keeping up with Young Workers 

This article was written by Marc Prensky, who is the Vice President for Personnel 
ofthe Bankers Trust Corporation [Ref 60]. Readers should therefore note that this paper 
was written by a businessman, and not a researcher. The author ıs one of many forward- 
thinking businesspersons that are trying to actively engage his workforce and get the 
most out of them. While his article does not provide any statistics on what should work, 
he is able to provide information on what does work (at least in his corporation). The 
author has been able to incorporate his methodology into a separate business (Corporate 
Gameware), which provides his software to other corporations for any task that they 
should desire to be taught. The synopsis is as follows: 

Our generation grew up with images being thrown at us at increasing levels of 
speed. My personal most hated list includes the ads where the camera angle kept 
changing. This is what advertisers thought we wanted, because it was already a part of 


our live. Video games change at ranges surpassing 60 images a minute, MTV videos 
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change even faster (surpassing more than 100 images a minute); action films go even 
faster. We have not only grown accustomed to these speeds, we have begun to accept 
these speeds as the standard for our entertainment; anything slower is deemed “dull.” 
Marc Prensky has rightfully pointed out that although we demand this speed for 
our entertainment and many other aspects of our lives, we do not demand it in our 
education. When a person joins a company, they are normally trained in traditional 
methods. This normally means guest speakers in suits, lulling the new recruits into a 
comatose state. Industry then began presenting videotapes, often allowing the individual 
to train at his or her own pace, watching guest speakers in suits.... The author has 
identified ten of the main cognitive style changes that can be made to improve training, 
and enhance work production. 
а. Dealing with a faster rate of speed 
There are very few professions or skills that have incorporated modem 
speed capabilities into the work environment (pilots, race car dnvers, and the like). Some 
corporations have been created to try to enhance individual worker speeds (such as 
Evelyn Wood’s speed reading), but for the most part, it is the technologies that are 
increasing in speed, and it is the only requirement of the worker that they be able to 
mentally process the information being presented to them. Several possible approaches 
include speeding things up via technology (such as by providing workers with the kinds 
of real-time data that financial traders use), installing faster infrastructures with fiber- 
optic cable and T-1 telephone lines, and creating new, MTV-style corporate videos (these 
work particularly well with young enlisted service members). Re-engineering systems 


and activities so that things simply move faster 1s another. 
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b. You can chew gum and walk at the same time! 

Despite what some of our elders told us, it 1s possible to do two things (or 
more) at once. While most of our older members of society cannot fathom how someone 
could get work done with a radio and/TV blaring, most of our generation thrives in such 
an environment. While it is true that we focus our attention on one source of data input at 
a time, 1t is also true that our brain 1s capable of allowing multiple inputs and processing 
them all (allocating attention in various orders). Media industry first began to take 
advantage of this when they began presenting more than one kind of information on 
video products (think CNN with its ticker tape news at the bottom of the screen). 
Prensky’s main point here is that managers need to take advantage of their workers 
capacity to take into multiple work inputs simultaneously versus yelling at their workers 
for doing it on their own! 

с. Random versus sequential access to information 

Today’s generation has grown up being able to have random access to 
media. It is kind of like our grandparents being able to read the end of the book before 
reading the beginning and/or middle of the book. There are times when they needed to 
do such, but normally, the best method was to go through the book sequentially. 
Prensky’s approach is to set up new information-delivery systems, such as corporate 
intranets, that let workers break out of the traditional boxes in which corporate 
information has been stored, and then to create tools to link this information to systems 
that provide logical and decision-making structure. Prensky states that the Intelink, is 
the best example of this concept. The Intelink is an intranet-based system in which 


information becomes universally available as quickly as it gets created. This allows 
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government employees at all levels to access the data they need sequentially, or 
randomly. 

a. A picture says a thousand words 

In older generations, you only read books with lots of pictures until you 
were actually able to read; then you read real books. Now, people are able to process 
data much quicker with the use of icons (especially when searching for items on the 
internet). Progressive generations are becoming more comfortable with images, and it 
could be said that they even prefer them (think about the community information displays 
found in Hermann Hall and outside the Welcome Center). We are entering an era where 
it is often best to present pictures until the user is ready for text. Further, we can keep 
users attention longer if that text is incorporated into graphical displays. Prensky has 
developed numerous games that actually train their users while they are being 


entertained. See figure 3.13 for an example of a game trainer. 





е. A new paradigm for communications 

Previous generations communicated information via physical meetings, or 
telephone calls. These communications were normally unsuccessful should one of the 
key persons not be available (think about the commercial where nobody knows who 
called the meeting, or the many times you have played “telephone tag”). People entering 
today’s workforce no longer need to have this connection; we can leave messages for 
each other (or for anyone in the world) via bulletin boards, chat rooms, emails or 
voicemails. We could literally carry out a precise conversation and never actually hear 
each other’s voices! 

А Active versus Passive Engagement 

Previous generations grew up having to go sequentially through manuals. 
If you didn’t follow bicycle instructions step-by-step, you would fail in the assembly. 
Current generation users only use the instructions as a last resort. This processing has 
come about from video games where it was easier (and normally faster) to leam a game 
through active tnal-and-error than through passive engagement. New employees have 
much less tolerance in the workplace for passive situations such as lectures, corporate 
classrooms, and even traditional meetings. They are truly the Nike generation of “Just 
Do It” versus the RTFM (“Read the F---ıng Manual”) generation. 

2. Approaching work as play 

Today’s generation has grown up on video games. They are used to 
solving problems in a way that is enjoyable to them. Prensky believes that this 
generation approaches work in a play like fashion. Successful business comes about 


from achievement, winning, and beating competitors are all very much part of the ethic 
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and process. Thus Prensky has tried to link business with video games (“Business has 
lots of content, but no engagement; Games have lots of engagement, but no content, if 
you put the two of them together, you make a new way to learn that makes sense” --- 
Quote from Prensky’s interview with FastCompany.com. 

Prensky has therefore developed a series of 12 software packages that he 
uses to incorporate corporate training games. The game interfaces look just like many of 
the games that we use for our daily entertainment. Nobody should forget the photo of the 
DoD employee of the year (back around 1995), which showed the woman next to her 
computer, which had an active game of Solitaire displayed. РгепзКу 5 games not only 
include Solitaire, but also Doom and Quake! Prensky’s games inject important business 
subjects into the games, which allow the users to both have fun, and learn about the 
subject at hand. For example, in the Doom game, the monsters throw problems at the 
user, and the user has to provide an appropriate solution; if he fails to do so, the user dies! 

h. Payoff versus patience 

Previous generations always had the proverbial “Gold Watch” as the long- 
term goals for their careers; they knew that continually performing at a constant rate 
would yield a satisfactory retirement. Today's generation desires immediate 
gratification; they want to see the seeds of their performances now, not in 30 years. 
Many companies have already begun to meet these expectations by providing their 
employees a “piece of the action,” through programs such as “pay-for-performance,” 
stock options, and other equity-like compensations. Companies have also been more apt 
to provide funding for internal “spin-off” companies, especially when they are internet- 
based. 
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i. Fantasy versus Reality 

IBM used to be famous for its gray. pinstriped suits. Most corporations 
had formal attire, and strict guidelines for employee work spaces. While this provided a 
professional work environment for clients, it stymied how comfortable the employees 
actually were. Many modern businesses (especially the computer giants like Netscape or 
Microsoft) allow their employees to set up their workspaces as they desire. They also 
allow their sections to create fantasy names for themselves, and try to make the work 
environment as much of a “home-way-from-home” as they can. The end result is that 
employees tend to voluntarily stay, and work longer hours. The corporations yielding on 
standards have seen the results of higher work production. 

j. Technology being perceived as a friend versus a foe 

Previous generations perceive technology as a threat; many lost their jobs 
due to technological advancements. Generation X changed with technology; generation 
Y has embraced ıt. To the younger generation, the computer is a friend, and a 
companion. Being on the Internet 1s a part of their lives, and how they communicate with 
each other. 

In this day of scarce employees, young people are getting the opportunity 
to choose who they want to work for; often times, that selection is not based upon how 
much money they will make, but rather upon what technology the will be provided. 
Employers need to understand that putting a young adult in an office without a computer 
is like taking the proverbial fish out of the water; they will seek out the job that keeps 


them connected. 
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For additional information about Twitchspeed, please refer to the 


following websites: http://www. fastcompanv.convonline/17/videogames.html. or 


http://www newsweek.com/nw-srv/1issue/22 98b/printed/us/bz/bz0622 1.htm 





D. CAS CONCEPTS AND DEFINITIONS 


The following concepts and definitions come from a variety of sources, though 
most come from two sources. The primary source 1s Jacques Ferber’s book’ entitled 
"Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence.” Many 
researchers, to include the NPS MOVES curriculum see this book as the foundation for 
agent-based simulations. Ferber released the book in his native language (French) in 
1998; and in English in 1999 [Ref 42]. The second significant source 1s Professor John 
Hiles of the Naval Postgraduate School. Other sources include Robert Axelrod, John 
Holland, Brian Arthur, and Andrew Ilachinski. 

1. Computerized/Chaotic Object 

A computerized object is computer code designed to both store and manipulate 
information. The object contains variables that store the information that 
defines/represents the object. The object also contains functions/methods that initialize, 
change, and retrieve the pieces of information stored in the variables. The functions 
normally send messages to each other. When objects are agents, the agent-object will 
have autonomy of action, rich interiors, and random errors that occur in the messages sent 
between the functions. 

DR Artificial Agents (AKA “Agents”) 

Agents are computerized objects that have the capability to act independently and 


autonomously. While an agent can operate on its own, it is the interactions between 
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agents that dnve this thesis. The agents socialize with each other, each with its own 
variable values, and differing goals. Sometimes the agents choose to cooperate with each 
other, while other times they may choose to compete, or even antagonize each other. 
There are two types of agents, physical and virtual. The physical agent operates within a 
simulated copy of the real world; a good example of this is Craig Unrath’s helicopter 
reconnaissance simulator [Ref 61]. The virtual. agent operates in an “unsituated” 
environment, which is one without a counterpart physical environment. 
Ferber elaborates that an agent 1s an entity that: 
e 1scapable of acting in an environment 
e can communicate directly with other agents 
e is driven bya set of tendencies/goals 
e possesses resources of its own 
е is capable of perceiving its environment 
е has only a partial representation of its environment 
e possesses skills and can offer services 
e may be able to reproduce itself 
e has behaviors which tend toward satisfying its objectives 
3. Kenetics 
Kenetics, as defined by Jaques Ferber, is the ability to plan, design and create 
universes or organizations made up of artificial agents which are capable of acting, 
collaborating in common tasks, communicating, adapting, reproducing, perceiving the 
environment in which they move and planning their actions to fulfill objectives defined 


either extrinsically or intrinsically on the basis of a general objective of survival. 
D 


Kenetics has the following aims: 


4. 


To define a scientific discipline which takes account of interaction 
between agents as the basis for understanding the functioning and 
evolution of systems. 

To define the various forms of interaction, such as cooperation, 
competition, collaboration, obstruction, etc., and link them to the issues of 
auto-organization, performance, or the survival of the systems. 

To outline the main mechanisms giving rise to auto-organization, such as 
grouping, specialization, distribution of tasks and resources, coordination 
of actions, conflict resolution, and so on. 

To define operational models of these interactions by describing the 
functioning of agents and of multi-agent systems. 


Complex Adaptive System (CAS) 


A CAS is a special living or man-made system comprised of many autonomous 


sub-components that interact with (and effect) each other, and display changing 


systematic behaviors. Впап Arthur wrote in his paper entitled “Bounded Rationality” 


that a CAS is a system whose behaviors fluctuate between the extremes of being rigid 


(purely linear) and (purely) chaotic [Ref 41]. 


Over time CAS system behavioral patterns form, leading evaluators to believe 


that the system can be modeled using standard linear models; however, the behavior 


pattern will eventually dissipate, displaying behaviors in which traditional modelers 


would state that no model could ever possibly represent the CAS behavior (e.g. small 


changes in inputs causing extreme changes in outputs, or a change in one spot having a 
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ripple affect upon the whole system). One interesting thing to note about any CAS is that 
despite continuous replacement of components, the CAS will still remain! 

Prior to the 1980’s, most scientists had the belief that the study of systems 
displaying chaotic behaviors was futile. Even those scientist who conceptualized that 
certain systems occasionally formed patterns, were overwhelmed with their complexity, 
and thus put their study into the “too hard to do” category. However, the technological 
revolution that began in the 1980’s has made it feasible for scientist to now conduct many 
of these studies due to the tremendous increase in computing power. 

Since that time, scientists have found that many systems previously believed to be 
chaotic were actually just CAS.’ Classic studies on CAS now include the study of ant 
colonies, bees, forests, and Marine combat behaviors! Many scientists now believe that 
all CAS can now be modeled, and that the best way to do so is with a Multi-Agent 
System. 

5. Multi-Agent System (MAS) 

A MAS is a computer program that contains more than one agent inside of a 
simulated environment. The program provides the environment for the agents to operate 
in, and interact with, each other. A MAS is different from traditional system programs in 
that the emphasis 1s on the interactions between agents, and not on the parts of the 
system/environment. Analyzing the interactions allows the user to observe systematic 


emergent behaviors that would never be detected in a top-down systems approach. 
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Ferber defines a MAS as system that is comprised of the following six sub- 


elements: 


6. 


E: An environment that is a space that generally has a volume; this 15 
where the agent exists, and contains the agent's world. 

O: A set of objects that are situated within the environment. Basic objects 
are manipulatable, but not capable of manipulating other objects or the 
environment. Objects can be perceived, created, destroyed, and modified. 
A: An assembly of agents, which are themselves, advanced objects. They 
are the active entities of the system. --- Agents are objects with intent! 

R: An assembly of relations that link objects (and thus agents) to each 
other. 

Ops: An assembly of operations that make it possible for the agents to 
perceive, produce, consume, transform and manipulate objects. 
Operators/Laws: Operators are the world's actions upon objects, and the 
world's reactions to the actions of agents. Agents also use operators upon 
objects. Laws are the world's reaction to the agents' operators. 


MAS studying a CAS 


There are three types of MAS studies upon CAS. The first 1s “Micro-Social 


Studies,’ which is the study of interactions between agents, the forms of interactions, and 


linkages between agents or objects. The second is “Group Studies,” which is the study of 


processes or organizations that exist within the population (like military retention). The 


third is “Population Studies,” which is the study of the dynamics of large numbers of 


agents, structures and evolution. 
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Zi Communicating MAS 
Often time's agents within a MAS can learn from each other through direct 
observation of their neighbors (agents located within an agent's sensing range). 
Additionally, the agents can be designed to send messages directly to each other; in such 
cases, the agents are called “acquaintances,” and the MAS is labeled as a Communicating 
MAS. 
8. Purely Communicating MAS 
Purely Communicating MAS (PCMAS) are also known as “un-situated MAS.” 
The acquaintance agents within a PCMAS are also known as “software agents.” These 
MAS do not have synthetic environments that mimic naturally occurring environments. 
These MAS only have conceptual environments that may or may not be described 
pictorially. The purpose of these environments is to provide a situation where the 
acquaintance agents can interact (communicate) with each other. PCMAS are good for 
modeling systems that rely primanly upon information distribution. 
Ferber defines a PCMAS agent as having the following characteristics: 
e Exists in an open computing system (assembly of applications, networks, 
and heterogeneous systems) 
e Communicates with other agents 
e Driven by aset of its own objectives 
е  Possesses resources of its own 
е Maintains only a partial representation of other agents 


*  Possesses skills (services) which it can offer to other agents 
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e Ехегіѕ behavior tending towards obtaining its objectives, taking into 
account the resources and skills available to it and depending on its 
representations and on the communications it receives. 

9. Situated MAS 

A situated MAS (SMAS) is a multi-agent system whose agents are positioned in a 
synthetic environment. SMAS are models of systems whose components actually exist in 
physical space. If a STEVE simulation had several agents operating together, it would be 
a good example of a Situated MAS. 

10. Purely Situated MAS 

A purely situated MAS (PSMAS) is a situated MAS in which the agents do not 
communicate by sending messages but only by the propagation of signals. The PSMAS 
contains agents that represent physical entities, which are called purely situated agents. 

Ferber describes purely situated agents as having the following characteristics: 

• [s a physical entity or simulated computing entity 

e Is situated in an environment 

e Is driven by a survival/satisfaction function 

е Possesses resources of its own 

е Is capable of a limited perception of its environment 

e Has practically no representation of its environment 

e  Possesses skills 

e Can reproduce/pass on Its traits to other follow-on agents 

е Behaves with a tendency that fulfills its survival/satisfaction function, 


taking into account the resources, perceptions, and skills available to it 
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11. Environments 

For situated MAS, the environment is a physical space or simulation of a physical 
space. The agents within the environment are capable of perceiving the environment, 
objects within the environment, and some of the other agents that also live in the 
environment. The agents are capable of manipulating each other, and the objects within 
the environment. The net result of all manipulations is the changing of the environment 
itself. When the MAS environment is changeable, it 1s called a reactive MAS. 

12. Levels of Organization 

Ferber descnbes three levels of MAS organization, the Micro-Social Level, the 
Group Level, and the Global Society Level. The purpose of Micro-Social Level MAS 15 
the study of the interactions between agents, and of the relationships that form between 
pairs or groups of agents. The purpose of Group Level MAS is the study of 
organizations, and the role/relationships that form/emerge from within the organization. 
The purpose of Global Society Level MAS is the study of large numbers of agents, their 
interaction with the environment, and the environmental evolution that occurs. 

13. Emergence 

Emergence is the property of evolving MAS. It is the behavior that the 
environment exudes due to the interactions of the agents with other agents, objects, and 
the environment itself. The emergent properties of MAS often result in the whole system 
being greater than the sum of its parts. MAS Systematic output is often studied to 
determine whether emergent behaviors forms from the component agents collective 


behaviors. 
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14. Cognitive Agents 

Cognitive agents (CA) are agents that have goals and explicit plans which allow 
them to achieve those goals. In comparison to reactive agents, CA are considered to be 
the more sophisticated agents; this 1s because they have considerably more information 
storage and functionality which allow them to operate independently of other agents. 


э? 


There are two principle types of cognitive agents, “intentional agents,” and “module- 
based agents.” 

а. Intentional Agents 

Intentional Agents are the predominant type of CA. Their name implies 
that they have explicit goals that motivate their actions (thus everything they do is 
intentional). They are also referred to as “rational agents” because the intelligent and 
intentional actions they take are normally seen as rational steps towards achieving their 
internal goals. 

b. Module-Based Agents 

Module-based agents are also known as “reflex cognitive agents.” 
Although they are agents themselves, they provide a secondary mission within MAS, and 
are seen as aides to the primary agents. Their sole purpose for existence in MAS 15 10 
provide information to the primary agents, or to provide a specific interaction with the 
primary agents. Module-based agents are perhaps the agents most closely associated 
with basic objects. 

I5 Reactive Agents 


Reactive agents (RA) are less sophisticated than cognitive agent because they are 


intentionally designed to have limited capabilities in the areas of data storage, and 
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functionality. These limitations cause RA”s to be cautious in their decision-making. 
Thus RA”s do not plan out what they will do in the future; they simply react to significant 
events that occur to them and their environment. The limitations also cause the RA’s to 
interact with each other in order for each to develop better environmental and situational 
awareness (while each RA only has limited knowledge, collectively they have 
considerable knowledge). 

It is important to note that while CA can operate independently, RA’s need 
acquaintances. Thus the behaviors that emerge from RA’s are truly systematic, and not 
individual behaviors from specific agents. RA’s are sub-divided into two main 
categories, “drive-based agents,” and “tropistic agents.” 

а. Drive-Based Agents 

Drive-based agents react to specific events occurring within their 
environment. These reactions are in accordance with one or more rules that are designed 
to achieve specific goals for each agent (rules will be different from agent to agent). If no 
events are occurring, the agents typically remain in an unchanging state. Drive-based 
agents have what is called “hysteretic memory;” this means that the agent is capable of 
remembering what has happened before (and thus it remembers how it reacted to a 
situation before, and whether or not that reaction was appropriate). 

b. Tropistic Agents 

Tropistic agents are reactive agents that continually respond to the stimuli 
coming from within their environment. Their decisions are based purely upon the current 
state of the environment, and thus their decisions will continue to be the same until 
something changes the environment. Events do not have direct effects upon tropistic 
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agents; their only effects are those that occur indirectly through manipulation of the 
environment. These agents have (appropriately named) “tropistic memory,” which 
means that they do not remember situations from the past, how they reacted to those 
situations, or if the previous decisions it made were appropriate --- they simply react to 
stimuli. 

16. Interaction 

Interaction is the process of agents dealing with other agents (to include module- 
based agents). The agents learn from each other, and also influence the decisions that 
each may make in the future. Some interactions result in positive influences, while others 
result in negative influences, or in neutral influence. Interactions also allow agents to 
develop and learn about their environment from each other. This learning allows the 
agents to do more than they could have by themselves. While each agent develops on a 
personal level, the system as a whole changes, emerging into a new state. 

17. Адаргапоп 

Adaptation is the process whereby an organism fits itself into its environment. 
There are two different levels of adaptation, individual and systematic. Individual 
adaptation occurs when an individual agent changes itself in response to feedback that it 
receives from its peers and/or environment. Like humans, the agent learns over time, and 
adapts how it conducts itself. Some agents will learn to be successful in their 
environment, while others will not. 

Systematic adaptation is similar to the adaptation of living organisms (as 
discussed by Richard Dawkins [Ref 6]). However, in this case, the adaptation must be 


driven by a routine within the synthetic environment. The routine evaluates the success 
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of each agent, and the traits that those agents possess. The routine then creates 
replacement and/or additional agents with combinations of the most successful traits. 
The routine may also throw in random variations of traits in order to include the concept 
of mutation into the MAS. 
18. Agents Abilities 
Each agent will have differing levels of the following agent properties: 
e Action: Able to modify their environment 
e Communicate: Signals or messages between agents 
e Intention: Intrinsic or autonomous 
e Resources: The wherewithal to take action 
e Partial Knowledge: Point of view; they know about some of the world 
they are in (not all, not none) 
e Capability: Skills and services 
e Feedback: Persistence and sometimes reproduction. 
e Reproduce: Ability to create follow-on agents (though not always) 
19: Agent Domains 
Each agent has its' own domain. It may also belong to a larger domain, or may 
even belong to several levels of domains. Each domain may direct the agent to have a 
different “intent.” The agent wants what satisfies its needs, but it may belong to other 
domains that call for it to satisfy external needs (of those domains). The end result may 


be that the agent’s behaviors work towards satisfying all or none of these needs. 
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20. Agent Knowledge 

Normally, agent-objects only have partial knowledge of their environments. This 
allows them to grow and learn from each other. In general, agent knowledge is based 
upon what it has encountered, and what it has tried to do itself. What an agent believes to 
be true may or may not be what is globally/absolutely true. In MAS, global knowledge is 
generally considered god-like; thus perfect knowledge within a system. is normally only 
associated with external observers. 

21. Stimulus — Response (S-R) 

Stimuli cause responses in an agent, and are thus the events that cause learning. If 
the agent benefited from the response, the S-R is “positively reinforced,” if not, it is 
"negatively reinforced" (Encoding). Multiple stimuli can hit an agent at the same time 
(often the same stimuli hitting different senses). 

22. | Credit Assignment 

Credit assignment 1s the process by which an agent awards or punishes its internal 
rules (raises or lowers the value of that specific rule). If a rule allows the agent to take 
advantage of a good situation, then it is rewarded. If a rule stops an agent from taking 
that advantage, it is punished. If a rule causes an agent to take a bad action, it 1s 
punished. If a rule stops the agent from taking a bad action, it is rewarded. Over time, 
the agent will rely most heavily upon the rules with the highest values/weights. À good 
example of this rule system 1s that of a frog. If a rule says to eat a fly, it is rewarded. If 
the rule says do not eat the fly, it is punished (the frog didn't get to eat a good meal). If 
the rule says eat a bumblebee (and the frog gets stung), the rule is punished. If the rule 


says not to eat the bumblebee, it 1s rewarded (for protecting the frog from pain). 
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23% Genetic Algorithms (GA) 

Genetic Algorithms are formulas for MAS that reproduce agents based upon 
previous “parent” agents. These MAS take the most successful traits from current agents, 
and base them as the foundation for future agents. The three basic types of selection are 
creation (e.g. by God), random, and natural (letting nature take its course). The key 
mechanisms for GA are crossover (genes mixing), inversion (mutation), and 
chromosomes (Predictors). The chromosomes are packages containing many (at least 
two) genes/rules (called “alleles”). Thus offspring receive chromosomes with genes from 
their parents, some of which may have been altered via the key mechanisms. The 
representation of rules in the new generation 1s roughly proportional to the success that 
they had in the previous generation. The result 1s the creation of a new set of agents, 
most filled with successful combinations of rules, and occasionally some (.01% to 
.001%) with new combinations of previously successful rules. This allows the successful 
solutions (and derivatives of them) to continue, while less successful ones die out. 

24. | Goals and Rules 

Goals are what each agent is trying to accomplish. In situated MAS like ISAAC, 
they include such things as advancing, or capturing the enemy flag. In unsituated MAS, 
they may include maximizing happiness, or obtaining the lowest price. Actions are the 
means by which agents attempt to achieve their goals. Examples of actions include the 
situated actions of taking the shortest route/safest route, and the unsituated actions of 
avoiding confrontations, or interacting with as many seller-agents as possible. Quite 
often, the current conditions within an environment will have a significant impact upon 


what actions an agent will take. 
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Goals must be measurable in order to see 1f they have been achieved. Goal 
measurements may be absolute, or have intermediate measurements. One of the best 
ways to determine whether a goal has succeeded or not, 1s to introduce feedback into the 
model. An example of this is intermediate commander agents reporting back information 
to global commander agents. 

25: Relationships . 

Relationships form organizations. In the case of retention, each servicemember 
agent has relationships with his/her family, peers, mentors, and its chain of command. 
Thus the relationships are the connections between agents. Each relationship will have 
different roles. To family agents, the servicemember agent is a father/mother; to peers, 
the role will be that of friend; to the chain of command, the servicemember agent may be 
both a leader and a follower. Each relationship may cause an agent to change its goals. 
Relationships can change over time, thus affecting an agent's goals. 

A good example of changing goals is that of a single servicemember who gets 
married and then has children. This agent has added roles, which may or may not affect 
his/her behavior. The amount that roles affect behavior 1s based upon the weight that the 
agent applies to that specific role. The primary job of wartime leaders is often 
convincing servicemembers to weigh their role as protectors of our country more than 
they weigh their other roles, and life itself. 

26. Malevolent Agents (MalAgs) 

Malevolent Agents are agents that are thrown into MAS for the explicit purpose 
of causing the primary agents to deal with turmoil. MalAlgs may not always be 


observable by the primary agents, and may wait until crucial moments in a scenario 
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before they unleash their harmful effects. Utilizing MalAgs ensures that agents and 
humans (in the loop) plan for the worst possible scenarios occurring (Murphy’s Law). 

27. | Mobile Agents (AKA Aglets) 

Mobile agents are agents that transport themselves from one computer system to 
another. These agents are very specialized, and have specific missions given to them by 
their ^owners," such as seeking out the best airfare for a vacation trip, or finding new 
parts for an aging computer. Aglets from various owners therefore meet and negotiate 
business in cyberspace (anywhere from within a local area network (LAN) to the Internet 
itself). The aglets thus allow their owners to concentrate on other business. The aglet 
negotiations are somewhat similar to what their owners would do; each negotiates within 
a given set of parameters, and either come to an agreement, or choose to do business 
elsewhere (routing themselves dynamically). Each aglet keeps its goals secret from other 
agents so that they cannot be taken age of by the other aglets. Once a mobile agent 
makes an agreement (or after it has met its maximum search time), it reports back to its 
owner. 

Aglets communicate by sending messages to each other. The method is called a 
“push and pull scheme. Pushing occurs when an aglet wants to get something from 
another aglet. Pulling occurs when the other aglet wants something from the first agent. 
The environment surrounding the negotiations 1s called a “place.” 

Aglets have many advantages over traditional programs. Some of the advantages 
are that they reduce the overall network load, can overcome network latency, encapsulate 


protocols, are adaptive, and are robust. 
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Е. SUMMARY 


The Department of Defense has an increased challenge in analyzing Military 
Personnel Retention during times of continuous change. The Technological Revolution 
and US economic boom have jointly created such an unstable environment here at the 
start of the 21% Century. Current linear retention models arc proving less than optimal, as 
the Services continue to be caught off guard by extreme changes in service member 
retention behavior. The military therefore needs new models to help deal with the 


paradigms it is now facing. 


Although technological advances are partially to blame for the current retention 
situation, they can also be used to help adapt and overcome the crisis. One of the major 
benefits of the Technological Revolution is the great increase in computer processing 
speeds. This advancement has made possible the study of complex system behaviors 
previously thought to be too complex for modeling. Many scientists are now finding 


order in systems previously believed to be purely chaotic. 


Researchers have begun to utilize MAS for military research purposes. Andrew 
llachinski has led the research through his ISAAC simulations. A number of military 
officers have begun to utilize MAS for their thesis research, to include one on military 


retention (Gaupp’s afore-mentioned PICAS simulation for Air Force aviators). 


CAS terminology can be complex, and confusing. As this research field is 
relatively new, different researchers may call the same things by different names. 


Jacques Ferber is the first researcher/professor to release a book that detailed key 
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CAS/MAS terminology and concepts. Ferber’s book on MAS is seen by the NPS as the 


most complete and comprehensive source on CAS/MAS. 


This thesis is a merger of the previous research analyzed in the areas of Military 
Personnel Retention and Complex Adaptive Systems. The author believes that the 
literature researched provides an adequate foundation for the creation of a military 
personnel retention simulator capable of simulating any military occupation specialty 


within the Armed Forces. 
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IV. MODEL DEVELOPMENT 


"Nothing goes by luck in composition. It allows for no tricks. The 
best you can write will be the best you are." --- Henry David Thoreau 


A. FILE HEIRARCHY 


There are twelve significant java files used in the MPRS project. Three of the files 
(MPRS, Simulation, and Environment) are the code that drive the simulation and provide 
the environment for the agents to operate in (the MPRS file creates Simulations which in 
turn create the Environment in which the agents operate). Four additional files (MOS 
Setting Editor, Simulation About Box, Agent Dialog and Environment Dialog) provide 
the user with the opportunity to fine tune simulations by adjusting parameter values. One 
file (Data) is a storage class used to pass information between the driver files. The 
remaining four files (Agent, Spouse, Child, and Civilian Friend) are all agent classes that 
represent service members, the families, and friends. It should be noted that the service 
member agents also play the roles as mentors and military frends to other service 
member agents. 


B. FILE DESCRIPTIONS 


Each of the twelve java files named above serves a different purpose within the 
MPRS simulator. The following summaries briefly descnbe each of the files, and its 
purpose. 

1. MPRS 

This is the main MPRS executable file. It determines whether the user wishes to 


run a demonstration or a user-defined simulation through a series of Java dialog boxes. If 
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the user desires to set his/her own parameters, this file extracts that information and 
inserts it into the simulation. Data passed between files is stored in a storage “Data” 
object. Regardless of the type of simulation run, this file creates the appropriate 
operating environment, and then executes the simulation. Once the simulation is 
complete, this program determines whether or not the user wants to run another 
demonstration, his/her own simulation, or to shut down the program. 

7 Simulation 

The Simulation class determines what branch of service the user wants to 
emulate, and then creates the frame that will contain the simulation during its execution. 
It then creates an environment object, and adds it to the simulation. Simulation is also the 
file that determines what the user wants to do whenever he/she clicks on the main MPRS 
panel. In particular, Simulation is responsible for taking action whenever the user clicks 
on the control buttons (start, pause, resume, clear, and environment controls). When one 
of these buttons is clicked upon, Simulation takes the appropriate action. 

3. Environment 


> 


This is the environment in which the servicemen agents "live." As this is an 
unsituated simulation, there is no physical environment. All agents therefore exist in this 
communications space (which is made to look like the life-cycle of a service member 
careers for users to observe what the agents are doing). Environment creates all of the 
various types of agents, and assigns service member agents their mentors, friends, and 
families (the make-up and number of family members being driven by random 


generations based upon the information contained in the Data object passed into it). Each 


agent is represented by a filled in oval, which is colored to reflect the rank of the agent. 
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The vertical location of the agent depicts how happy they are with their careers (the 
higher up on the panel they are, the happier they are; the unhappier they are, the lower 
they are on the panel. 

If agents get too unhappy, and are at a point where they can choose to get out, 
they will do so by resigning/ETSing (Expiration of Time in Service), either of which is 
signified by the service member agent exploding into a red ball of fire at the bottom of 
the panel). This is the problem that the military is trying to avoid (we do not want service 
members choosing to depart the service). As time goes by, the agents will move from the 
left side of the panel to the nght, signifying time elapsing (each simulation step is 
equivalent to one month of a service member’s career). 

When the agents come upon solid vertical lines, it is time for them to be 
considered for promotion. Environment selects agents for promotion based upon their 
previous job performance (total scores from the evaluations that they receive each year). 
Agents with the best ratings are promoted (which may or may not be reflected by their 
location on the panel). Agents that are not promoted get passed over (which is also 
reflected by their exploding into a red ball of fire). If too many service members choose 
to get out on their own, all remaining service members will be promoted (which is not 
what the military wants). 

Environment represents time one month at a time by moving all of the agents 
three pixels to the nght. It tracks agents until they depart the service, or have 22 years 
time in service (at which time it removes the agents from the simulation). Agent 
happiness can be determined based upon their location on the panel. A dotted honzontal 


line depicts average happiness; agents above that line are happy, with those below it 
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being unhappy. Agent "inclination" (direction their happiness is going) is determined by 
the slope depicted by the direction that each agent takes. Clicking on a given agent will 
bring up a data box about that agent, which includes information about the inclination. 

When an Environment creates its agents, 1t emplaces them into a vector called the 
“Agent Vector.” Whenever an agent departs the service, it is moved into a second vector 
called the “АСАР (Army Career and Alumni Program) Vector." At the conclusion of the 
simulation, information is extracted from the agents in the ACAP vector, and then 
provided to the user on the DOS Command Line. This data should be transferred to a 
statistics package such as S-Plus or MS Excel for analysis. 

4. MOS Setting Editor 

This class creates a dialog box that allows the user to edit the parameters being 
used in the simulation. The user is provided the dialog box just before the simulation is 
executed. The user is shown the current settings that will be used should he/she choose 
not to make any changes. This is done by extracting the values currently stored in the 
Data object (that will be sent into the simulation). If the user chooses to change the 
parameter values, he/she need only type over the current settings. When the user hits the 
"okay" button, the parameter values are sent back to the Data object (over-writing the old 
values contained there). 

5. Simulation About Box 

This AboutBox 1s based upon an AboutBox created by LT Mike Dickson (who 
used JBuilder3 to create his version). This file simply allows the user to receive MPRS 
development information whenever they select the appropnate menu bar item from the 
top of the MPRS panel. When selected, the user is provided with information pertaining 
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to who created the MPRS, and what version it is in. This allows multiple users to ensure 
that they are both using the same version of the MPRS. After June 2001, updates of the 
MPRS will be available at the website www. familyfrench.net. 

6. Agent Dialog 

The Agent Dialog Box 1s used to display all pertinent information about a specific 
service member agent. It is activated by either left or nght clicking onto a specific agent 
on the MPRS panel (it is easiest to do this after hitting the “pause” button). When an 
agent is selected, that agent’s dialog box will appear (an example of the Agent Dialog 
Box can be found below at figure 4.1). All information presented is extracted from the 
data fields within the service member agent objects. 

The dialog box is broken down into four main areas: Agent Data, Career Data, 
Family Data, and Agent Attitude. 

e The Agent Data section provides information about the selected service 
member agent. It states the agent’s number, its rank/pay grade, its color 
(which graphically represents its rank), its fictitious social security 
number, and its military occupation skill (MOS). 

e The Career Data section provides the user information about the career 
success that the agent 1s having up to this point in the simulation. In 
addition to branch of service and career path (officer/warrant officer 
/enlisted), this area tells the user what year group the agent belongs to 
(what year it started its career), how long 1t has been in the service, how 
long it has been at its current rank, and most importantly, what its 


evaluation reports have been like up to this point in its career. 
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Center of Mass (COM) means that the agent 15 having an average career; 
above COM (ACOM) means that the agent is having a superb career; 
below COM (BCOM) means that the service member is having a poor 
career. When agents come up for promotion, their ratings will determine 
who gets promoted, and who gets put out of the service (it should be noted 
that 1f too many service members choose to get out of the service on their 
own, then all remaining agents will be promoted). 

The Family Data section provides information about the service member 
agent that may have considerable impact upon whether it opts to get out of 
the service or not (the agent’s family). This section shows the user the 
marriage status of the agent, and how many total dependents the agent has 
(these numbers are based upon information provided into the simulation 
by the user; random numbers determine which agents have which types of 
families). Each family member is an actual agent (though module-based), 
whose happiness level affects the happiness of his or her service member 
agent. 

The last section depicts the Agent Attitude. The attitude of a service 
member agent is determined by its family, career, and the environment. 
The section depicts current satisfaction (how happy the service member 
agent 1s), and also the inclination of that happiness (is it getting better or 
worse?). As in real life, an improvement in any of these areas may result 


in a positive effect upon the service member agent. 
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This Dialog 1s instantiated by the user clicking on the "Environment Controls" 
button on the MPRS panel. The dialog is how the user changes the major environment- 
wide factors that affect the everyday lives of the agent servicemembers. The settings 
include the perceived Chain of Command concern, the perceived Department of Defense 
concern, the perceived civilian-military pay gap, and the perceived Operations Tempo. 
All of these factors are prefaced with “perceived” because it 1s perception that dnves 
servicemembers’ beliefs about reality. These factors should all be set based upon current 


perceptions within the applicable area being studied. 
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АП four factors start of as being neutral/average. Each factor has a slider bar that 
enables the user to change the values. NOTE: The values can be changed while the 
simulation is running; however, users should make in-run changes only to see that the 
changes take effect immediately. If the user would like to compare runs at different 
settings, the user should make changes prior to starting the simulation run (and allow 
each simulation to complete its run without mid-cycle manipulations). 

8. Data 

This class 1s a data storage class used by the main files to store numerous 
parameter values. The data object is then sent between the files as one large parameter, 
allowing ease of use, and readability by the user. Only one Data object is used in any 
given time. Methods inside the class consist primarily of "getter" and "setter" methods 
for manipulating the parameters stored within the object. 

Ө; Agent 

Agent is the file that actually completes the service member agents when the 
Environment Class “creates” them. These agents are the key objects in the simulation as 
they represent individuals and their decision-making. Each agent contains all of the 
personnel information necessary to manage its career. The agent keeps track of its rank, 
years of service, time at its current rank, time obligations to the service, how happy it is 
with the service, and whether it 1s getting happier or unhappier (inclination). The agent 
also keeps track of how many family members it has, and takes the happiness of those 
family agents into consideration when determining its own happiness levels. Each 
service member agent also has a senior agent as a mentor, military friends (other service 


member agents), and civilian friend agents. As with family members, each service 
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member agent takes all of these other agents' happiness levels into consideration when 
they evaluation their own happiness level. 

If an agent gets out of the service, it remains actively engaged with all other 
service member agents that it has relationships with (that are still in the simulation). If 
the service member agent finds happiness outside of the military, it will negatively affect 
the happiness of his service member agents that are still in the military; if such an agent 
does not find happiness as a civilian, it positively affects its service member agent friends 
still in the military. 

When an agent retires or gets out on its own accord (ETS/Resignation), it stores 
all of the information about why it got out (data such as 1ts unhappiness level, years of 
service, pay grade, and family situation). That information is extracted upon the 
completion of the simulation, and is provided to the user for analysis via the DOS 
Command Line. 

10. Spouse 

The role of the spouse in this simulation is simply to affect the service member 
agent like a real spouse would affect their real service member. When instantiated, the 
spouse is assigned a random happiness; over time, the spouse agent's happiness is 
affected by the things that keep the service member agent busy and/or away from the 
family. 

Spousal inclination is not changed easily, which can be either a good or bad thing 
for the service. If the spouse agent is happy, it will take time and many negative events 
to change that happiness. On the other hand, if the spouse agent is unhappy, it will take 
time and many positive events to change the spouse's negative feelings about the military. 
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If the service couple has no children, the spouse will have a slower rate of change 
in regard to current circumstances; however, once the service couple has children, the 
spouse's inclination towards the military will change quicker, representing the spouses 
protection over his/her children. If the service member gets divorced, the agent will 
either become a single parent (with custody of dependent children), or become single 
again. 

11. Child 

Child represents one non-spouse dependent of a service member agent. The 
initial happiness of a child agent is normally distnbuted; some will initially love the 
military, others will initially hate it, and yet most will be apathetic about the military. 

As time elapses, the child agents will be affected mostly by the operations tempo 
of the service. The more the service deploys troops, the more the child agent will dislike 
the military; if deployment rates remain at a normal/average rate, the child will not get 
unhappy with the service; 1f deployment rates go below average, the child will prefer that 
their parent remains being a service member over their parent becoming a civilian. 

As with Spouse agents, happiness or unhappiness of child agents does not change 
drastically. The happiness is derived by previous happiness and the current situation. 
However, as long as operations tempo remains high, the happiness of children agents will 
get worse (thus making the happiness of service member agents worse --- though at a less 


significant level). 
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125 Civilian Friend 

The role of the civilian friend agents in this simulation is also to affect the service 
member agent. These agents represent what the service member agent could be doing if 
it opted to get out of the military to become a civilian. It should be noted that the 
happiness of civilian friend agents is inversely related to the happiness of the service 
member agents (if the civilian is happy, then the military agent is made unhappier, if the 
civilian agent is unhappy with its life, then the military agent is happier with the life it is 
experiencing). 

The happiness of civilian friends is driven by two major parameters. The first is 
the employment rate in the civilian sector. The employment rate is used to determine 
whether a given civilian friend 1s employed or not. If the agents are in technical fields, 
the odds are that the civilian agent is well employed (whether a given agent is employed 
or not is governed by the employment rate dictated by the user, and a random number 
generator). Users should utilize career field employment rates if such rates are available; 
if not, the user should input the current national employment rate, and understand that 
their career field may have a different rate. 

The second major parameter that affects a civilian friend agent's happiness is the 
perceived pay gap (an environment factor). If the perceived pay gap is high, then the 
civilian friend agent's perception is that they are earning much more money in the civilian 
sector than they would in the military (and are thus they are happy about that). If the 
perceived pay gap 15 low or negative, then the civilian friend agents will develop a type 
of "jealousy" about their military peer's earning more money than they are making. 
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С. RUNNING THE MPRS MODEL 


Users can start the MPRS in one of two ways. The easiest way 1s by using a Java 
IDE such as Jbuilder or Forte. However, executing the program from this mode will not 
allow the user to receive error messages or the output generated by the program. 
Therefore, the preferred method is to execute the program from the MS-DOS Command 
Prompt. In order to execute the main MPRS program file, it 1s necessary to be in the 
DOS directory immediately above the MPRS directory. At that time, the user can 


execute the file by typing “Java MPRS.MPRS”. 


Once the program has started, a series of dialog boxes will ask the user for input 
necessary to run the simulation. The first dialog box simply requests the user’s name for 
future reference. The second request is for the user to decide whether or not to run a 
demonstration (novice users should utilize this option in order to get a better 
understanding of how the program works before inputting their own parameters). Should 
the user choose to observe a demonstration, it will run, and then the program will ask if 
the user would like another simulation. The user can then choose to observe another 


demonstration, or move on to creating his/her own simulations. 


When the user chooses to create a simulation, the program requests the detailed 
information necessary to delineate branch of service, career path (officer, warrant officer, 
or non-commissioned officer), and military occupation specialty (the specific job skill). 
The user then decides how long the simulation should run (in months). The user ıs then 


ready to input personnel management data specific to the chosen servicemembers. 
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In order to identify specific settings, the user 1s provided a panel with 15 data 
fields divided into three categories (an example of the panel is found at figure 4.2 below). 
The first category contains the number of personnel authorized at each skill level. The 
second category contains the number of years that service members average at each skill 
level. The third category contains current group-wide personal information about the 
service members. This simulation runs best on a computer that contains a 700-megahertz 
processor (or better). If the simulation appears to run too slow on a slower computer, the 
user can adjust the simulation by reducing the number of agents (and authorizations) by a 


constant factor (such as by dividing all numbers by 10). 
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Figure 4.2: Military Occupation Specialty Input Panel 
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Personal data items currently being taken into consideration include family data 
(marriage rate, divorce rate, single parent rate, and overall dependent rate), and the 
unemplovment rate of the service member's civilian counterparts. These are the factors 


that most affect the retention behaviors of service members. 


Once the user has updated all of the group-wide personal data fields, the MPRS 
simulation panel appears (some users may need to adjust the size of the simulator to 
better fit the available screen). At this time, the user may begin to utilize the MPRS 
control buttons found at the bottom of the panel. A copy of the panel is found at figure 


4.3 below. 
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Figure 4.3: — MPRS Panel 
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The MPRS panel is design to emulate the career progression that service members 


go through during their career. Each service member is depicted by a single dot (an 


agent) on the MPRS panel. Service member progressions are depicted by their 


movement from left to right across the screen. Service members change color if they are 


promoted, and explode off of the screen if they are passed over for promotion, or choose 


to depart the service on their own. 


The panel itself 1s modeled after the Army officer career progression model used 


to counsel officers on their career progression (an example of which can be found at 


figure 4.4). The panel includes six points of interest which are highlighted in figure 4.5: 


Entered onto Active Duty (EAD) Line. This is the first vertical line from 
the left. The line signifies the point in time when service members enter 
onto active duty. 

Promotion Lines. These are all remaining full vertical lines. It is at these 
lines where the service promotes or releases service members from active 
duty. 

20+ Years Line. This is the only dashed vertical line. This signifies the 
point in a service member’s career where they will be able to retire. 

Ranks (Column Headers). These are the titles that the service members 
hold while at the given pay grades. 

ETS line. This is the lower horizontal line. This signifies where service 
members are eligible to leave the military, and choose to do so. 

Control Buttons. Clicking on these buttons allow the user to run and 


manipulate the MPRS. 
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There are five control buttons at the bottom of the MPRS panel, those being Start, 


Pause, Resume, Clear, and Environmental Controls: 


Start 1s used to initiate new simulations. Hitting this button during a 


simulation run will cause a reset to occur. 


Pause is used to temporarily halt a simulation (particularly useful is vou 
want to reset multiple environmental controls without their making 


immediate individual changes). 


Resume is used to restart a paused simulation (pushing Start at this point 


will not effect the desired result). 
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e Clear is used to clear the current simulation, and eliminate all data that 


had been collected by that simulation (to include deleting the contents of 


the ACAP Vector). 


e Environment Controls is the last button on the panel. The button is used 
to manipulate the environmental controls in order to adjust the factors 


controllable by the DoD (these controls can be adjusted at any time, to 


include their being adjusted while the simulation 1s running). 


When the Environment Controls button 1s selected, a dialog box appears which 
allows the user to manipulate the levels at which the user believes the given service 
members perceive the Civilian-Military Pay Gap, the Operations Tempo, the Chain of 
Command Concer, and the DoD-level Concern (the President, down to the particular 


service agency). An example of the environment controls panel 1s found at figure 4.6 


below. 





Percieved Pay Gap 


PA 7241 AA 
-50-45-40-35-30-25-20-1519-5 0 5 101520 25 30 3540 4559 


Pay Gap - 0.0 
Percervec COC Concem 
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-50-45-40-35-30-25-20-1512-5 0 5 1015 20 25 30 35 40 4550 


CoC Coacem="00 


——————M HA 
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Figure 4.6: 


Environment Control Panel 
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D. DATA SOURCES 


Data used for the MPRS came from a variety of sources within the Department of 
Defense. The primary source for data for the MPRS was Professor William K. Krebs of 
the NPS. Dr. Krebs has been analyzing military retention over the last decade, and was 
the onginator of the idea for using agents to simulate military retention behaviors. His 
research with Professor Sam Buttrey (also of NPS) and the linear retention theses studies 
of his students during the 1990s were of great assistance for the creation of this thesis. 
Professor Krebs is currently working for the Federal Aviation Administration and is on a 


leave of absence from the NPS. 


Other sources for data for this thesis include the Defense Manpower Data Center 
(DMDC), the Army Office of the Deputy Chief of Staff for Personnel (ODCSPER), and 
the Army Research Institute (ARD. The DMDC was particularly helpful in providing 
information about what reasons service members are giving for getting out of the 
military, and what attitudes young Amencans have about the service. 


E OUTPUT ANALYSIS 


The MPRS simulation records the military careers of the agent service members 
throughout each simulation. Each agent is removed from the “active duty” vector when 
they depart the military (through being passed over for promotion or getting out on their 
own accord) and placed into another vector for storage (named ACAP for the Army 
Career and Alumni Program). Once the simulation is complete, all data from the ACAP 


vector is displayed in the DOS window in report format (see figure 4.7 for an example). 
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Copying this data into a spreadsheet program (such as MS Excel) allows the user to 


analyze the differences made when different factor levels are used. 


As stated in previous sections, the uniqueness of complex adaptive systems is that 
the output resulting from minor changes in inputs will result in the full spectrum from 
rigidity to chaos. The accuracy of the parameter settings will control how accurate the 
output relates to the real world. Although the scope of this thesis did not include data 
analysis, I provided the output features in order to provide feedback to users. Full data 
analysis is therefore assigned as future work for the author and follow-on graduate 


students. 


кие ee ae 


pr ЧИ 

100000043 |1LT 
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100000048 — (LT 
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100000063 
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Figure 4.7: MPRS Output 
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Е. SUMMARY 


The MPRS is an outstanding tool for helping personnel managers to think about 
the decisions that service members make, and the factors that effect those decisions. 
When properly fine-tuned to a specific MOS, the MPRS can help a personnel manager to 
understand the group-wide effects that service members have upon each other. More 
importantly, the MPRS helps personnel managers understand that retention behaviors can 
become non-linear, and that they must therefore be observant for extreme changes over 


short periods of time. 
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V. FUTURE WORK AND CONCLUSIONS 


"Ihe toughest thing about being a success is that you've got to 
keep on being a success." --- Irving Berlin 


The creation of this thesis and the MPRS model was a great learning experience. 
When I completed Version 1.0 of the model, I began briefing and demonstrating the 
MPRS in many forums (to include my Thesis Defense). During several of these events, I 
was able to obtain professional feedback from colleagues, military personnel managers, 
NPS faculty members, and simulations community representatives. Conversations with 
these persons also allowed me to discover the need for other improvements. 

During the last three months, I was able to implement several changes to improve 
the MPRS model; I therefore call this revision of the MPRS model “Version 2.0.” 
However, there were still several other quality recommendations that I was not able to 
implement prior to the deadline for publishing this thesis. This section describes some of 
the possible future enhancements that I will strive to make in the future. These changes 
will result in a second major revision of the model, which will be labeled “Version 3.0.” 
This improved version of the MPRS will be released on/about 1 July 2001, and will be 
available at www.familyfrench.net. This section also contains the overall conclusion for 


the thesis. 
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А. FUTURE WORK 


The following list encompasses changes that will provide users with improved 


MPRS simulations. 


1. 


There 


simulation: 


Add/Implement Additional Relationships and Parameters 


are several relationships and parameters that can be added to the 


The most important relationship not currently in the MPRS model is that 
of the leader-follower (employer-employee). These relationships often 
have a considerable impact upon the happiness of a service member. 
Right now this relationship is generally covered by the “Perceived Chain 
of Command Concern,” but I feel that it would be more effective if 
implemented as an agent relationship. 

A significant parameter that should be added is the service member’s 
gender. The retention behavior of females 1s different than that of males. 
This will also allow for the model of joint servicemember families. 
Another parameter that can be added is the service member's mission 
acceptance. In other words, the service member may or may not agree 
with the policies that he/she 1s enforcing. If the service member does not 
believe in the mission, he/she may decide to depart the service; if the 
service member does agree with the mission, it. will reinforce their 


happiness with being in the service. 


MS 


2. 


Improved Functionality 


Some MPRS functions can be improved: 


Currently, evaluation reports are given out on a random basis. It is my 
goal to add functionality that awards evaluations based upon added agent 
parameters that represent hard work, intelligence, and values. This change 
would result in the best service member agents getting the best 
evaluations. In turn, the best agents will be promoted, while the worst will 
be passed over for promotion. 

Promotion boards do not currently allow for early promotions (below the 
zone) or for late promotions (after the zone --- the year after a service 
member's peers were promoted). In scenarios where too many service 
members get out of the service (from one year group), the year group 
remains under-manned throughout the remainder of the simulation run. 
This is contrary to the reality of the services reaching down into later year 
groups, and early promoting members of junior year groups (which then 
causes a ripple effect until the deficit is made up). 

The model assumes that the same number of entry-level personnel enter 
the service each and every year. The model can be improved to 
manipulate the number of entries, to include setting a maximum limit 
(which would represent the reality of recruiting). This changing number 
of accessions would allow the above-mentioned early promotions to occur 


(as it would be the ultimate cure for the situation). 
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3. Remove Unnecessary Code/Streamline Code 

The MPRS was the very first java program that I have ever written outside of 
assignments from a Java programming course I sat in on. During the last year, I learned a 
lot about Java coding. As time went by, my coding became more efficient; however 
there are still sections of code that are wntten in very inefficient manners --- this code 
needs to be streamlined. Another challenge is that the project contains some code that no 
longer provides significant functionality, but is wrapped around part of code that does 
provide necessary functionality; removing this code will require time and patience. 

4. Add more Random Events (Mutations) 

Currently, the only random events that occur are the marnages and divorces of 
service members. It is my belief that more random events should be added. This 
includes such events as random deaths, police actions (arrests), and other judicial (e.g. 
chapter) actions. 

5: Improving the Java Panel/Environmental Display 

While the MPRS program provides statistical data after a given simulation run, 1t 
does not provide the user ongoing information during the actual processing. The model 
can be changed in order to display data on the sides of the main panel. This change 
would allow users to analyze events occurring to year groups as they process through the 
simulation (the ISAAC panel is a good example for this change). 

6. Significance Levels of Environment Controls 

Currently, each environmental control is weighed evenly. The four control each 
feed into the happiness of agents in two ways. First, each factor holds a weight of 12.5% 
of every agent’s happiness (for a total value of 50% of every agent’s happiness). Second, 
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the factors are used in varying degrees to determine the other 50% of a specific agent’s 
happiness. The weights of the factors can be changed to comparative values, and thus 
split up their 50% of the happiness weights in a way that better represents the real world). 

In order to make this change, a routine will need to be added that solicits from the 
users information pertaining to the importance of each factor. A good model for this 
concept is the Army’s Course of Action software (MAPS), which assigns weights based 
upon stated parameter significance. 
B. CONCLUSIONS 

This thesis discusses the current problems that the Armed Forces are having with 
military personnel retention, and how technology can affect it in both positive and 
negative ways. The MPRS model provided with this thesis is an example of technology 
assisting personnel managers through the use of multi-agent systems. The MPRS is an 
outstanding thinking tool for personnel managers, and is meant to supplement, not 
replace, current linear models. The continued improvements of this and other MAS 
retention models such as PICAS will help the military better understand the group 
behaviors of its service members during times of continuous change. This improved 
knowledge (if acted upon in a timely manner) will enable the military to shift its 
personnel policies in order to retain the necessary amount of personnel necessary to 


ensure our readiness goals. 
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